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Preface 


Goals  of  this 
Guidebook 


Guidebook 

Organization 


How  to  Use  this 
Guidebook 


In  this  guidebook,  we  provide  sponsors  of  acquisition  improvement  programs  and  their 
immediate  staff  with  guidelines  on  how  to  implement  a  software  acquisition  risk  man¬ 
agement  program  satisfying  the  goals  of  the  Acquisition  Risk  Management  (ARM)  Key 
Process  Area  (KPA)  of  the  Software  Acquisition  Capability  Maturity  Model® 
(SA-CMM®).  Brief  overviews  of  software  acquisition  and  the  SA-CMM  can  be  found 
in  Appendix  A,  p.  71  and  Appendix  B,  p.  75,  respectively. 

The  following  table  outlines  the  guidebook  organization. 

Component 

Purpose 

Chapter  1 

Provide  overviews  of  risk  management  and  the  ARM  KPA, 
and  list  recommendations  for  each  key  practice  of  the  ARM 
KPA. 

Chapter  2 

Provide  detailed  expansions  of  each  key  practice  within  the 
ARM  KPA.  Each  key  practice  is  described  to  help  readers 
understand  the  objective  of  the  practice  and  examples  are 
provided  to  help  readers  apply  Ae  practice  in  various 
situations.  The  concept  of  teaming  with  other  organizations 
to  cooperatively  manage  project  risks  is  also  explored. 

Appendix  A  - 
Software  Acquisition 
Overview 

Provide  a  short  introduction  to  softwzire  acquisition. 

Appendix  B  - 
The  Software 
Acquisition  CMM 

Provide  a  short  introduction  to  the  SA-CMM. 

Appendix  C  -  Risk 
Management  Methods 
and  Tools 

Describe  select  methods  and  tools  used  in  risk  management. 

Appendix  D  -  Selected 
Risk  Management 
Forms 

Provide  select  forms  used  in  risk  management. 

Appendix  E  - 
The  SA-CMM 
Appraisal  Process 

Describe  the  appraisal  process  used  during  an  SA-CMM 
appraisal  and  provide  sample  questions. 

Depending  on  the  individual’s  role  or  function  in  the  organization,  different  components 
of  this  guidebook  will  be  of  more  interest  than  others.  The  table  below  provides  a  sug¬ 
gested  way  to  navigate  this  guidebook  depending  on  that  role  or  function. 


CMM  and  Capability  Maturity  Model  are  registered  in  the  U.S.  Patent  and  Trademark  Office. 
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Role/Function 

Desire 

Guidebook  Com¬ 
ponent 

Acquisition  organization 
management 

(e.g.,  manager  above  the 
project  manager,  sponsor) 

Gain  general  understanding  of 
Acquisition  Risk  Management 

Appendices  A  &  B 

Chapter  1 

Project  management 

(e.g.,  project  manager, 
chief  engineer,  chief 
technical  officer,  division 
chiefs) 

Learn  what  Acquisition  Risk 
Management  is 

Appendices  A&  B 

Chapter  1 

Chapter  2 

Coordinator/developer  of 
Acquisition  Risk 
Management  process 

(e.g.,  technical  managers 
or  leads,  software 
acquisition  process  group 
members) 

Learn  what  Acquisition  Risk 
Management  is,  how  to 
interpret  the  KPA,  and  select 
alternative  methods  and  tools  to 
use  when  defining  a  risk 
management  process  for  a 
project 

Appendices  A  &  B 

Chapter  1 

Chapter  2 

Appendix  C 

Appendix  D 

Appendix  E 

Participant  in  Acquisition 
Risk  Management 

(e.g.,  engineers,  project 
officers,  matrixed  support) 

Understand  Acquisition  Risk 
Management  and  how  to 
participate  in  a  project’s 
defined  risk  management 
process 

Chapter  1 

Chapter  2 

Appendices  C  &  D 

To  fully  understand  the  guidelines  presented  in  this  guidebook,  readers  should  possess  a 
general  understanding  of  the  structure  and  content  of  the  SA-CMM  and  of  basic  risk 
management  terminology  and  principles. 

The  following  documents  were  used  extensively  to  develop  this  guidebook. 


Dorofee,  A.;  Walker,  J.;  Alberts,  C.;  Higuera,  R.;  Murphy,  R.;  &  Williams,  R.  Continu¬ 
ous  Risk  Management  Guidebook.  Pittsburgh,  Pa.:  Software  Engineering  Institute, 
Carnegie  Mellon  University,  1996. 

Fisher,  M.;  et  al.  Software  Acquisition  Capability  Maturity  Model  (SA-CMM) 

Version  1.02  (CMU/SEI-99-TR-002,  ADA362667).  Pittsburgh,  Pa:  Software  Engineer¬ 
ing  Institute,  Carnegie  Mellon  University.  Available  WWW 
<URL:  http://www.sei.cmu.edu/publicatlons/documents/99.reports/ 
99tr002/99tr002abstract.html>  (1999). 
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Overview 


Chapter  1 


Chapter  1 


Risk  Management 

This  chapter  introduces  readers  to  risk  management  and  the  Acquisition  Risk  Manage¬ 
ment  (ARM)  Key  Process  Area  (KPA)  of  the  Software  Acquisition  Capability  Maturity 
Model  (SA-CMM).  Recommendations  from  Chapter  2  are  summarized  here  for  easy 
reference. 


Section 

Risk  Management  Overview 
Acquisition  Risk  Management  Overview 
Summary  of  Recommendations 
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Chapter  1 
SccMion  I 


Overview 


Risk  Management  Overview 


This  section  introduces  risk  management  and  provides  an  overviev^  of  the  identify,  ana¬ 
lyze,  plan,  track,  control,  and  communicate  functions  vital  to  successful  implementation 
of  an  acquisition  risk  management  process. 


Section 

Risk  Management  Process 
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Section  l.l 


Section  1.1 


Risk  Management  Process 


What  Is  Risk? 


Risk  vs. 
Opportunity 


Risks  vs. 
Problems 


Risk  Example 


Problem 

Example 


Facilitates 

Communication 


There  are  a  number  of  definitions  of  the  term  risk,  but  none  is  universally  accepted. 
However,  all  definitions  of  risk  have  two  common  characteristics  [Kirkpatrick  92]: 

•  uncertainty:  an  event  may  or  may  not  happen 

•  loss:  an  event  has  unwanted  consequences 

The  SEI  uses  the  following  definition  of  risk:  Risk  is  the  possibility  of  suffering  loss 
[Dorofee  96]. 

Risk  and  opportunity  are  related.  Opportunity  for  advancement  can’t  be  realized  with¬ 
out  taking  a  risk.  In  this  case,  risk  should  not  necessarily  be  viewed  negatively,  because 
it  is  essential  to  making  progress.  The  key  is  to  balance  the  potential  negative  conse¬ 
quences  of  risk  against  the  potential  benefits  of  opportunity  [Kirkpatrick  92]. 

Example:  A  company  that  wants  to  increase  its  market  share  might  decide  to  assume 
more  risk  in  order  to  achieve  its  goal. 

As  defined  above,  risk  is  the  possibility  of  suffering  loss.  Notice  that  uncertainty  is 
associated  with  risk — an  event  may  or  may  not  happen.  When  a  negative  event  or  issue 
is  a  certainty,  it  is  considered  to  be  a  problem,  not  a  risk. 

The  software  for  a  system  being  acquired  must  be  developed  using  C++  and  object-ori¬ 
ented  (00)  technology.  The  contractor  selected  to  develop  the  software  has  experience 
in  the  application  domain,  but  has  little  experience  with  C++  and  00  development.  Per¬ 
sonnel  on  the  project  team  are  concerned  that  the  contractor’s  inexperience  with  C++ 
and  00  technology  will  affect  its  ability  to  develop  a  system  that  meets  the  performance 
or  functionality  requirements  within  the  defined  schedule. 

The  risk  is:  The  contractor  does  not  have  experience  using  C++  and  OO  technology  ;  the 
system  may  not  meet  its  performance  or  functionality  requirements  within  the  defined 
schedule. 

There  is  uncertainty  associated  with  whether  the  system  will  meet  its  performance  or 
functionality  requirements  within  the  defined  schedule — the  contractor  may  or  may  not 
meet  the  requirements.  Therefore,  this  is  a  risk. 

An  organization  is  acquiring  a  manufacturing  process  control  system.  The  contractor 
informs  the  project  team  that  during  system  integration  and  test,  the  process  control  sys¬ 
tem  was  found  to  crash  periodically. 

There  is  no  uncertainty  associated  with  the  crashing  of  the  process  control  system — it 
occurs  periodically.  TTierefore,  this  is  a  problem. 

Glass  provides  an  in-depth  study  of  major  software  engineering  projects  that  have  failed 
[Glass  98].  One  shared  characteristic  of  these  failures  is  the  inability  of  project  mem¬ 
bers  to  communicate  potential  problems  to  the  decision  makers  within  a  project.  When 
interviewed  after  the  fact,  72%  of  the  failed  projects  had  team  members  who  knew  early 
in  the  development  process  about  the  details  of  potential  problems  that  eventually 
caused  havoc,  while  on  only  19%  of  these  projects  the  management  team  shared  the 
same  insight.  Risk  management  allows  team  members  to  discuss  potential  problems  in 
a  structured,  non-threatening  manner  providing  insight  to  decision  makers. 


Chapter  1 
Section  1.1 


Overview  of  a 
General  Process 


SEI  Risk 

Manaigement 

Paradigm 


Paradigm 

Functions 


There  are  many  models  for  managing  risk.  A  systematic  risk  management  process  must 
have  a  set  of  steps,  or  functions,  that  must  be  performed  to  manage  project  risks.  In  gen¬ 
eral,  it  must  provide  a  way  to  identify  project  risks,  to  evaluate  and  prioritize  risks,  to 
develop  plans  designed  to  mitigate  the  most  important  project  risks,  to  monitor  the 
progress  of  the  plans  and  their  effectiveness  in  reducing  risks,  and  to  take  additional  cor¬ 
rective  actions  if  necessary. 

The  SEI  risk  management  paradigm,  shown  below  [Van  Scoy  92],  defines  a  systematic 
process  for  managing  a  project’s  risks.  The  paradigm  consists  of  a  number  of  functions 
that  are  performed  as  continuous  activities  throughout  a  project’s  life  cycle. 

Note:  The  SEI  risk  management  paradigm  is  one  systematic  process  that  can  be  used  to 
manage  risks.  Other  processes  exist,  but  they  are  not  described  in  this  guidebook. 


The  functions  of  the  SEI  risk  management  paradigm  are  outlined  in  the  table  below 
[Higuera  93]  and  expanded  in  the  next  section.  Managers  must  rely  on  the  experience 
and  expertise  of  their  personnel  to  effectively  manage  risks;  the  knowledge  derived  from 
participation  in  an  activity  as  well  as  the  unique  skills  of  team  members  provide  manag¬ 
ers  with  additional  information  that  they  might  not  have  had  otherwise.  All  relevant  risk 
data,  including  decisions  made  and  actions  taken,  should  be  kept  in  a  repository,  because 
the  data  might  be  relevant  to  the  present  project  or  to  other  projects  within  the  organiza¬ 
tion. 
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Continuous 

Process 


Performing  the 
Risk 

Management 

Functions 


Paradigm  Function 

Purpose 

Identify 

To  search  for  and  find  risks  before  they  become  problems 

Analyze 

To  transform  risk  data  into  information  that  can  be  used  to 
aid  decision-making.  The  impacts,  probabilities,  and 
timeframes  for  risks  are  evaluated,  and  the  risks  are 
classified  and  prioritized. 

Plan 

To  transform  risk  information  into  planning  decisions  and 
mitigation  actions  and  to  implement  the  mitigation  actions 

Track 

To  monitor  risk  metrics  and  mitigation  actions  to  determine 
if  the  plan  is  on  schedule  as  well  as  if  the  mitigation  plan  is 
effective  in  reducing  the  risk 

Control 

To  make  informed,  timely,  and  effective  decisions  regarding 
risks  and  their  mitigation  plans 

Conununicate 

To  provide  information  and  feedback  about  the  risk  man¬ 
agement  process,  mitigated  or  watched  risks,  and  emerging 
risks.  Sources  for  risk  information  may  be  either  internal  or 
external  to  the  project 

Communication  is  an  enabler  of  the  other  paradigm  func¬ 
tions. 

A  risk  will  typically  progress  through  the  paradigm  functions  sequentially,  but  the  risk 
management  functions  are  performed  continually  (i.e.,  risks  are  managed  continuously 
throughout  all  phases  of  a  project),  concurrently  (i.e.,  mitigation  plans  for  risks  are 
developed  and  tracked  while  new  risks  are  being  identified  and  analyzed),  and  itera¬ 
tively  (i.e.,  a  mitigation  plan  for  one  risk  may  be  used  to  identify  another  risk)  through¬ 
out  a  project’s  life  cycle  [Dorofee  96].  Acquisition  risk  management  requires  sustaining 
constant  vigilance  and  managing  risks  routinely  throughout  all  phases  of  the  project’s 
life  cycle. 

The  risk  management  functions  need  further  definition  in  order  for  a  project  to  put  these 
concepts  into  practice.  The  next  section  expands  the  definition  of  each  function  and 
provides  components  that  can  be  used  by  a  project  to  define  a  risk  management  process. 
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Identify 


Analyze 


Section  1.2 


Identify,  Analyze,  Plan,  Track,  Control,  and 
Communicate 


Risk  identification  is  a  process  where  uncertainties  and  issues  about  a  project  are  trans¬ 
formed  into  tangible  risks,  which  can  be  described  and  measured.  Everyone  on  a  project 
is  responsible  for  identifying  risks.  The  following  table  describes  the  components  of 
risk  identification  [Dorofee  96].  Techniques  used  to  identify  risks  include  structured  or 
unstructured  brainstorming,  peer-group  interviews,  and  voluntary  reporting.  The  meth¬ 
ods  and  tools  used  to  support  identification  are  found  in  Appendix  C,  p.  79. 


Component 

Description 

Capture  statement  of 
risk 

Capturing  a  statement  of  risk  includes  considering  and 
recording  the  conditions  that  are  causing  concern  of  a 
possible  loss  to  the  project.  A  brief  description  of  the 
perceived  consequences  resulting  from  the  conditions  is  also 
included  in  a  statement  of  risk. 

Capture  context  of  risk 

Capturing  the  context  of  a  risk  involves  recording  additional 
information  regarding  the  circumstances,  events,  and 
interrelationships  within  the  project  that  supplements  the 
risk  statement.  Context  provides  more  detail  than  is 
presented  by  the  risk  statement. 

Risk  analysis  is  a  process  in  which  risks  are  examined  in  detail.  The  purpose  is  to  deter¬ 
mine  the  extent  of  the  risks,  how  they  relate  to  each  other,  and  which  ones  are  the  most 
important.  Personnel  who  have  the  appropriate  knowledge,  expertise,  and  background 
to  effectively  deal  with  risk  information  are  responsible  for  evduating,  classifying,  and 
prioritizing  the  risks. 

Example:  On  one  software  acquisition  project,  when  project  personnel  identify  risks, 
they  are  responsible  for  estimating  the  risks’  attribute  values  (see  table  below)  as  well  as 
the  risks’  classification  (see  table  below).  The  technical  leads  on  the  project  examine 
the  risks’  attribute  values  and  classifications  and  make  any  necessary  changes.  The 
technical  leads  are  also  responsible  for  prioritizing  their  teams’  risks. 

The  following  table  describes  the  components  of  risk  analysis  [Dorofee  96].  The  meth¬ 
ods  and  tools  used  to  support  analysis  are  found  in  Appendix  C,  p.  79. 


Component  Description 

Evaluate  Evaluating  the  attributes  of  a  risk  involves  establishing 

•  impact:  the  loss  or  effect  on  the  project  if  a  risk  occurs 

•  probability:  the  likelihood  that  a  risk  will  occur 

•  timeframe:  the  time  period  during  which  action  will  be 
required  to  mitigate  the  risk 
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Plan 


Component 

Description 

Classify 

Classifying  risks  requires  grouping  risks  based  on  their 
shared  characteristics.  The  groups,  which  can  also  be  called 
classes  of  sets,  show  the  relationships  among  the  risks.  Risk 
classification  can  be  used  to  help  identify  duplicate  risks  as 
well  as  to  help  simplify  a  list  of  risks. 

Prioritize 

Prioritizing  risks  involves  the  following: 

•  partitioning  risks  or  sets  of  risks  based  on  the  “vital  few” 
sense  [Juran  89]  to  separate  those  that  are  most  important 
from  the  rest 

•  ranking  the  most  important  risks  or  sets  of  risks  based 
upon  a  criterion  or  set  of  criteria  established  by  the  project 

The  product  of  risk  prioritization  is  a  ranking  of  the  most 
important  risks  to  the  project,  known  as  a  “top  N”  list  [Dor¬ 
ofee  96]. 

Planning  is  a  process  whereby  decisions  are  made  about  what  should  be  done  with  a 
risk.  The  results  of  planning  are  risk  action  plans  for  individual  risks  or  sets  of  related 
risks.  Personnel  who  have  the  knowledge,  expertise,  background,  and  resources  to 
effectively  deal  with  risks  are  responsible  for  developing  their  plans.  In  general,  the 
goal  of  planning  is  to  answer  the  following  questions: 

•  Is  it  my  risk?  (responsibility) 

•  What  can  I  do?  (approach) 

•  How  much  and  what  should  I  do?  (scope  and  actions) 

The  following  table  describes  the  components  of  risk  planning  [Dorofee  96],  The  meth¬ 
ods  and  tools  used  to  support  planning  are  found  in  Appendix  C,  p.  79. 

Component 

Description 

Assign  responsibility 

Assigning  responsibility  for  planning  requires  a  project 
manager  or  a  designated  person(s)  to  review  and  under¬ 
stand  the  risks  and  to  determine  what  to  do  with  them. 

There  are  three  choices  in  determining  responsibility  for 
risks: 

•  Keep  the  risk. 

•  Transfer  the  risk  upward  within  the  organization  or  to 
another  organization. 

•  Delegate  the  risk  within  the  organization. 
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Track 


Component  Description 

Determine  approach  Determining  an  approach  for  planning  a  risk  involves 

making  a  decision  about  the  type  of  plan  that  will  be 

required. 

•  Is  enough  information  known  about  the  risk?  If  the  answer 
is  no,  then  develop  a  research  plan  to  get  the  required 
information. 

•  If  the  risk  becomes  a  problem,  can  the  impact  of  the 
consequences  be  accepted?  Or  can  the  risk  be  more 
efficiently  addressed  at  a  future  time?  If  the  answer  is  yes, 
then  accept  the  risk,  expend  no  further  resources 
managing  it,  and  document  the  reasons  for  accepting  the 
risk  (acceptance  rationale). 

•  If  the  risk  can’t  be  accepted,  is  it  necessary  to  take 
immediate  action?  If  the  answer  is  yes,  then  mitigate  the 
risk  by  developing  and  implementing  a  mitigation  plan. 

•  Is  there  a  mitigation  action  that  can  or  needs  to  be  taken? 
Or  can  the  risk  be  accepted?  If  the  answer  is  no,  then  the 
risk  must  be  watched  and  tracking  requirements  must  be 
developed  (e.g.,  metrics  must  be  tracked). 

Note:  The  metrics  required  to  track  watched  risks  and 
mitigation  plans  are  defined  during  planning. 

Defining  scope  and  actions  involves  answering  the  follow¬ 
ing  questions  when  developing  a  mitigation  plan: 

•  How  complex  will  the  mitigation  be? 

•  How  should  it  be  documented? 

•  What  is  the  strategy? 

•  What  are  the  tasks? 

There  are  generally  two  types  of  mitigation  plans,  based  on 
the  nature  of  the  risk,  complexity  of  the  plan,  and  available 
resources: 

•  action  item  list  for  less  complex  mitigation  (one  or  more 
actions) 

•  task  plan,  including  schedules  and  budgets  for  complex 
sets  of  actions 


Define  scope  and 
actions 


Tracking  is  a  process  in  which  risk  data  are  acquired,  compiled,  and  reported  by  the  per- 
son(s)  responsible  for  tracking  watched  and  mitigated  risks.  The  metrics  gathered  dur¬ 
ing  tracking  are  defined  during  planning  and  are  presented  to  decision  makers  in 
tracking  documents  or  presentations.  The  information  is  then  used  to  make  control  deci¬ 
sions  about  watched  risks  and  mitigation  plans.  The  following  table  describes  the  com¬ 
ponents  of  risk  tracking  [Dorofee  96].  The  methods  and  tools  used  to  support  tracking 
are  found  in  Appendix  C,  p.  79. 
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Control 


,  Component 

/ 

Description 

Acquire 

Acquiring  risk  data  includes  all  of  the  steps  associated  with 
collecting  information  about  and  updating  the  values  of  risk 
metrics  for  watched  and  mitigated  risks.  The  purpose  of  the 
information  is  to  track  the  progress  of  watched  risks  and  risk 
mitigation  plans. 

Compile 

Compiling  risk  data  involves  analyzing,  combining, 
calculating,  and  organizing  data  for  a  given  risk  to  monitor 
the  progress  and  effectiveness  of  a  mitigation  plan  or  to 
monitor  changes  in  watched  risks. 

Note:  The  reporting  requirements  determine  how  project 
personnel  compile  the  data. 

Report 

Reporting  involves  communicating  status  information  about 
risks  and  mitigation  plans  to  decision  makers  and  team 
members.  Communicating  risk  information  can  be 
accomplished  with  written  reports  (using  either  paper  or 
electronic  media)  or  oral  presentations.  The  delivered 
reports  and  presentations  summarize  the  data  that  were 
analyzed  and  organized  and  are  used  by  decision  makers 
during  control. 

Control  is  a  process  in  which  a  decision  maker  analyzes  the  data  contained  in  tracking 
reports,  makes  a  decision,  and  implements  the  decision.  The  person  who  has  account¬ 
ability  for  a  risk  normally  makes  the  control  decision  for  that  risk.  The  following  table 
describes  the  components  of  risk  control  [Dorofee  96].  The  methods  and  tools  used  to 
support  control  are  found  in  Appendix  C,  p.  79. 

Component 

Description 

Analyze 

Analyzing  risk  data  includes  examining  project  data  for 
trends,  deviations,  and  anomalies.  The  goal  is  to  achieve  a 
clear  understanding  of  the  current  status  of  each  risk  and 
mitigation  plan  relative  to  the  project. 

Decide 

Making  a  decision  requires  using  tracking  data  to  determine 
how  to  proceed  with  project  risks.  Four  basic  decisions  with 
respect  to  risks  can  be  made: 

•  replan 

•  close  the  risk 

•  invoke  a  contingency  plan 

•  continue  tracking  and  executing  the  current  plan 

Execute 

Executing  a  decision  is  the  process  where  control  decisions 
are  implemented.  Making  changes  to  plans  requires  a  return 
to  planning,  while  taking  predefined  contingency  actions 
and  continuing  to  track  risks  requires  a  return  to  tracking. 
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Communicate  Risk  communication  deals  with  two  subjects  that  people  don’t  normally  communicate 

well:  probability  and  negative  consequences.  Managers  need  to  establish  a  culture 
where  risks  are  identified  and  addressed  as  a  part  of  everyday  business  and  where  risk 
information  is  viewed  positively  and  rewarded.  Successful  risk  communication  surfaces 
relevant  issues  and  potential  problems,  allows  information  to  be  exchanged  within  and 
between  all  project  levels,  values  the  individual  voice,  and  preserves  non-attribution  and 
trusted  use  of  all  risk  information.  Communication  is  an  enabler  of  the  other  paradigm 
functions  and  ensures  that 

•  Risks  and  their  mitigation  plans  are  understood. 

•  Risk  information  is  visible  to  all  project  members. 

•  Appropriate  attention  is  applied  to  risk  information. 

•  An  effective,  ongoing  dialog  between  the  manager  and  the  project  team  is  established. 
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Acquisition  Risk  Management  Overview 

What  Is 

Acquisition  Risk 

Management 

(ARM)? 

ARM  is  a  process  where  risks  are  managed  throughout  the  software  acquisition  life 
cycle  (see  Appendix  C,  p.  79).  It  is  a  two-part  process  [Marciniak  90]. 

•  Early  in  the  life  cycle,  the  risks  associated  with  the  acquisition  of  the  system  are 
identified  and  analyzed,  and  an  approach  to  mitigate  the  high-priority  risks  are 
incorporated  into  the  software  acquisition  plan. 

•  A  process  to  continually  manage  risks  throughout  the  software  acquisition  life  cycle  is 
integrated  into  the  project’s  defined  software  acquisition  process. 

When  Does  ARM 
Begin  and  End? 

Acquisition  risk  management  begins  with  the  process  of  defining  the  system  need  and 
continues  until  the  acquisition  has  been  completed.  Risk  must  be  considered  even  in  the 
earliest  stages  of  system  development,  such  as  determining  business  objectives  and 
developing  alternative  approaches  to  meet  those  objectives.  Likewise,  risk  must  be  con¬ 
sidered  through  user  acceptance  and  transitioning  maintenance  of  a  system  to  a  support 
organization.  Acquisition  risk  management  is  a  vital  part  of  the  entire  software  acquisi¬ 
tion  process. 

Purpose  of 
Acquisition  Risk 
Management 

The  purpose  of  acquisition  risk  management  is  to  identify  risks  at  the  earliest  possible 
time,  adjust  the  acquisition  strategy  to  manage  the  high-priority  risks,  and  implement  a 
risk  management  process  to  manage  risks  throughout  the  acquisition  life  cycle  [Marcin¬ 
iak  90]. 

Why  Manage 

Risk? 

Employing  risk  management  can  help  managers  identify  potential  problems  and  take 
action  to  prevent  the  problems  from  occurring.  When  managers  continually  manage 
risk,  they  can  avoid  disasters  and  prevent  costly  rework  [Boehm  89]. 

Acquisition  Risk 
Management 

Key  Process  Area 

The  goals  and  common  features  of  the  Acquisition  Risk  Management  Key  Process  Area 
(see  Appendix  B,  p.  75)  are  listed  below  [Fisher  99].  Each  of  the  goals  and  common 
features  will  be  examined  in  more  detail  in  Chapter  2  of  this  guidebook. 

Goal 

Description 

Goal  1 

Project-wide  participation  in  the  identification  and  mitigation  of  risks  is  encouraged 
and  rewarded. 

Goal  2 

The  project  team’s  defined  software  acquisition  process  provides  for  the 
identification,  analysis,  and  mitigation  of  risks  for  all  project  functions. 

Goal  3 

Project  reviews  include  the  status  of  identified  risks. 

Common  Feature 

Description 

Activity  1 

Software  acquisition  risk  management  activities  are  integrated  into  software 
acquisition  planning. 

Activity  2 

The  Software  Acquisition  Risk  Management  Plan  is  developed  in  accordance  with 
the  project’s  defined  software  acquisition  process. 
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Common  Feature 

Description 

Activity  3 

The  project  team  performs  its  software  acquisition  risk  management  activities  in 
accordance  with  its  documented  plans. 

Activity  4 

The  project  team  encourages  and  rewards  project-wide  participation  in  the 
identification  and  mitigation  of  risks. 

Activity  5 

Risk  management  is  conducted  as  an  integral  part  of  the  solicitation,  project 
performance  management,  and  contract  performance  management  processes. 

Activity  6 

Software  acquisition  risks  are  analyzed,  tracked,  and  controlled  until  mitigated. 

Activity  7 

Project  reviews  include  the  status  of  identified  risks. 

Commitment  1 

The  acquisition  organization  has  a  written  policy  for  the  management  of  software 
acquisition  risk. 

Commitment  2 

Responsibility  for  software  acquisition  risk  management  activities  is  designated. 

Ability  1 

A  group  that  is  responsible  for  coordinating  software  acquisition  risk  management 
activities  exists. 

Ability  2 

Adequate  resources  are  provided  for  software  acquisition  risk  management 
activities. 

Ability  3 

Individuals  performing  software  acquisition  risk  management  activities  have 
experience  or  receive  required  training. 

Measurement  1 

Measurements  are  made  and  used  to  determine  the  status  of  the  acquisition  risk 
management  activities  and  resultant  products. 

Verification  1 

Acquisition  risk  management  activities  are  reviewed  by  acquisition  organization 
management  on  a  periodic  basis. 

Verification  2 

Acquisition  risk  management  activities  are  reviewed  by  the  project  manager  on  both 
a  periodic  and  event-driven  basis. 
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Summary  of  Recommendations 


This  section  provides  a  summary  of  the  recommendations  for  the  key  practices  of  the 
ARM  KPA.  Refer  to  Chapter  2  for  a  detailed  discussion  of  each  key  practice.  Refer  to 
Appendix  E,  p.  101  for  an  overview  of  the  SA-CMM  appraisal  process  with 
typical  questions  directed  at  the  key  practices. 
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Activity  1 


Activity  2 


Software  acquisition  risk  management  activities  are  integrated  into  software  acqui¬ 
sition  planning. 

•  Project  teams  should  consider  risk  information  when  developing  software  acquisition 
strategies  and  plans. 

•  The  output  of  risk  identification  should  be  tangible  risk  statements  and  supporting 
context  information. 

•  Everyone  on  a  project  should  be  responsible  for  identifying  risks. 

•  The  output  of  risk  analysis  should  be  a  prioritized  list  of  the  project’s  risks  (i.e.,  a  top 
Nlist). 

•  Personnel  who  have  the  appropriate  knowledge,  expertise,  and  background  to 
effectively  deal  with  risk  information  should  be  responsible  for  evaluating,  classifying, 
and  prioritizing  the  risks. 

•  The  output  of  risk  planning  should  be  appropriate  plans  (i.e.,  research  plan,  acceptance 
rationale,  tracking  requirements,  or  mitigation  plan)  for  the  most  important  risks  (i.e., 
the  top  N  risks). 

•  Personnel  who  have  the  knowledge,  expertise,  background,  and  resources  to 
effectively  deal  with  risks  should  be  responsible  for  developing  their  plans. 

•  Risk  identification,  analysis,  and  planning  should  be  performed  early  in  a  project’s  life 
cycle  to  help  the  project  team  to  develop  its  software  acquisition  strategy. 

The  Software  Acquisition  Risk  Management  Plan  is  developed  in  accordance  with 
the  project’s  defined  software  acquisition  process. 

•  A  project  team’s  Software  Acquisition  Risk  Management  Plan  should  define  how  the 
risk  management  process  will  be  applied  to  the  project. 

•  The  Software  Acquisition  Risk  Management  Plan  should  be  based  on  the  project’s 
defined  acquisition  process. 

•  The  Software  Acquisition  Risk  Management  Plan  should  be  tailored  for  the  particular 
processes,  methods,  and  tools  used  by  the  project  team;  it  can  be  part  of  the  system- 
level  risk  management  plan,  part  of  the  project  management  plan,  or  a  stand-alone  plan. 

•  The  minimal  content  for  a  risk  management  plan  should  include 

•  introduction-defines  the  purpose  and  scope  of  the  risk  management  plan 

•  overview  of  processes-describes  all  risk  management  activities  and  their  relations  to 
other  project  management  activities  (see  Activity  1,  p.  25,  and  Activity  5,  p.  33) 

•  organization-defines  project  personnel  responsibilities  (see  Commitment  2,  p.  46)  as 
well  as  customer,  supplier,  and  co-developer  responsibilities 

•  process  details-describes  the  processes  and  procedures  for  systematic  risk 
management 

•  resources  and  schedule-documents  the  resources  (e.g.,  cost,  staff  effort,  equipment, 
software)  required  for  the  risk  management  process  (see  Ability  2,  p.  48) 

•  risk  documentation-defines  project  templates  and  forms,  database  tool 
specifications,  and  procedures  and  requirements  for  documentation 

•  The  current  list  of  risks  and  their  mitigation  plans  should  be  maintained  and  updated 
separately  from  the  risk  management  plan. 


Activity  3 


The  project  team  performs  its  software  acquisition  risk  management  activities  in 
accordance  with  its  documented  plans. 

•  The  project  team  should  follow  the  Software  Acquisition  Risk  Management  Plan. 
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Activity  4  The  project  team  encourages  and  rewards  project-wide  participation  in  the  identi¬ 

fication  and  mitigation  of  risks. 

•  Project  managers  should  pay  attention  to,  measure,  and  control  desired  risk  practices, 
activities,  and  behaviors. 

•  Teach  and  use  the  Condition  ->  Consequence  [Dorofee  96]  form  for  capturing  concerns 
and  issues  (potential  risks)  in  the  organization. 

•  Encourage  project  team  members  to  raise  concerns  and  issues  without  requiring  that 
they  have  a  solution  in  hand  for  them— but  make  sure  the  concerns  or  issues  are 
documented  in  the  project  risk  database  once  they  are  raised. 

•  Make  it  easy  to  document  an  issue,  but  hard  to  make  it  go  away;  reward  risk 
identification,  not  risk  closure. 

Activity  5  Risk  management  is  conducted  as  an  integral  part  of  the  solicitation,  project  per¬ 

formance  management,  and  contract  performance  management  processes. 

•  The  project  team  should  integrate  acquisition  risk  management  into  all  of  its  activities. 

•  The  contract  type  should  be  chosen  based  on  perceived  risk. 

•  The  project  team  should  identify,  analyze,  plan,  track,  and  control  risks  during  Project 
Performance  Management  activities. 

•  The  project  team  should  identify  the  risks  associated  with  the  contractor’s  activities. 

•  The  project  team  and  contractor(s)  should  enter  into  a  teaming  relationship  where  risks 
are  identified,  analyzed,  planned,  tracked,  controlled,  and  communicated  in  a  shared 
environment. 

Activity  6  Software  acquisition  risks  are  analyzed,  tracked,  and  controlled  until  mitigated. 

•  Project  teams  should  determine  whether  plans  are  being  executed  properly  and 
reducing  risk  by  gathering  and  examining  the  risk  metrics  that  are  defined  during  risk 
planning. 

•  The  output  of  risk  tracking  should  be  documents  or  presentations  highlighting  the 
relevant  tracking  data. 

•  Tracking  should  be  performed  by  the  person(s)  responsible  for  tracking  watched  and 
mitigated  risks. 

•  The  output  of  risk  control  should  be  decisions  (i.e.,  replan,  close  the  risk,  invoke  a 
contingency  plan,  or  continue  tracking  and  executing  the  current  plan)  which  are  then 
implemented  by  project  personnel. 

•  The  person  who  has  accountability  for  a  risk  should  make  the  control  decision  for  that 
risk. 

•  Risk  reporting  should  be  combined  with  routine  project  management  activities  (e.g.,  as 
part  of  a  weekly  or  monthly  project  status  update)  to  provide  the  project  team  with 
more  information  to  use  when  making  project  decisions. 

Activity  7  Project  reviews  include  the  status  of  identified  risks. 

•  Project  teams  should  sharpen  their  meeting  management  skills  to  improve  the 
productivity  of  reviews. 

•  Project  managers  and  team  members  should  develop  ground  rules  for  effective 
reviews. 

•  Project  managers  and  team  members  should  develop  and  enforce  acceptable  behavior 
guidelines  for  review  participants. 
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•  Integrate  risk  functions  (identify,  analyze,  plan,  track,  control)  into  the  review  agenda. 

•  The  project  team  should  select  methods  and  tools  that  communicate  the  status  of 
known  risks. 


Commitment  1 


Commitment  2 


Ability  1 


Ability  2 


Ability  3 


The  acquisition  organization  has  a  written  policy  for  the  management  of  software 
acquisition  risk. 

•  The  acquisition  organization  should  have  a  written  policy  on  software  acquisition  risk 
management. 

•  The  policy  should  include 

•  a  discussion  of  the  importance  of  identifying  risks  throughout  the  acquisition 

•  a  discussion  of  inter-organizational  risk  management  activities,  including  how  the 
project  team,  contractor(s),  and  end  user(s)  are  involved  in  risk  management 
activities 

•  how  risk  information  is  communicated 

•  designation  of  responsibility  at  the  acquisition  organization  level  (see  Commitment 
2,p.46) 

•  a  clear  statement  validating  acquisition  risk  management  as  a  positive  and  proactive 
part  of  software  acquisition 

Responsibility  for  software  acquisition  risk  management  activities  is  designated. 

•  Responsibility  for  software  acquisition  risk  management  activities  should  be  formally 
designated  at  both  the  acquisition  organization  and  project  levels. 

•  Projects  should  document  roles  and  responsibilities  in  the  Software  Acquisition  Risk 
Management  Plan  (see  Activity  2,  p.  28). 

•  At  the  acquisition  organization  level,  responsibility  should  be  designated  in  the  policy 
statement  (see  Commitment  1,  p.  45). 

A  group  that  is  responsible  for  coordinating  software  acquisition  risk  management 
activities  exists. 

•  The  acquisition  organization  should  ensure  that  adequate  personnel  are  available  to 
each  project  to  perform  the  acquisition  risk  management  activities. 

Adequate  resources  are  provided  for  software  acquisition  risk  management 
activities. 

•  The  acquisition  organization  should  ensure  that  projects  have  adequate  funding,  staff, 
equipment,  and  tools  to  perform  acquisition  risk  management  activities. 

•  Resources  required  to  perform  the  software  acquisition  risk  management  functions 
should  be  documented  in  the  Software  Acquisition  Risk  Management  Plan  and  should 
be  updated  as  necessary  throughout  the  project  to  reflect  changing  needs. 

Individuals  performing  software  acquisition  risk  management  activities  have  expe¬ 
rience  or  receive  required  training. 

•  The  acquisition  organization  should  provide  knowledgeable  personnel  to  project  teams 
to  perform  acquisition  risk  management  activities. 

•  A  written  plan  (e.g.,  the  project’s  Training  Plan)  should  specify  the  risk  management 
training  required  for  project  personnel  and  should  specify  the  training  schedule. 
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Measurements  are  made  and  used  to  determine  the  status  of  the  acquisition  risk 

management  activities  and  resultant  products. 

•  The  project  team  should  measure  the  status  of  individual  risks,  their  mitigation  plans, 
and  Ae  risk  management  process  to  determine  the  status  of  the  acquisition  risk 
management  activities. 

•  The  project  team  should  use  the  results  of  measurements  as  a  basis  for  project 
management  and  acquisition  organization  management  verifications  (see  Verification 
1,  p.  53,  and  Verification  2,  p.  54). 

Acquisition  risk  management  activities  are  reviewed  by  acquisition  organization 

management  on  a  periodic  basis. 

•  The  project  team  should  present  the  results  of  the  acquisition  risk  management 
activities  to  acquisition  organization  management  at  periodic  program  reviews. 

•  The  project  team  should  present  the  status  of  the  project’s  top  risks  and  mitigation 
plans. 

•  The  project  team  should  present  data  that  indicate  the  effectiveness  of  the  acquisition 
risk  management  process  (e.g.,  rate  of  identification  versus  rate  of  mitigation). 

Acquisition  risk  management  activities  are  reviewed  by  the  project  manager  on 

both  a  periodic  and  event-driven  basis. 

•  The  project  manager  should  review  and  participate  in  the  acquisition  risk  management 
activities. 
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Acquisition  Risk  Management  KPA 

This  chapter  provides  a  detailed  discussion  of  the  Acquisition  Risk  Management  KPA. 
Each  component  of  the  KPA  is  analyzed  and  expanded  and  includes  examples  and  fur¬ 
ther  definition.  Finally,  the  concept  of  managing  risk  with  other  organizations  is 
explored.  The  Acquisition  Risk  Management  KPA  identifies  risk  management  issues 
which  must  be  addressed  by  an  acquisition  organization  to  satisfy  the  defined  maturity 
level  of  the  SA-CMM.  It  includes  the  goals,  institutionalization  features,  and  activities 
required  to  implement  risk  management  in  an  acquisition  organization. 
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Goals 


The  acquisition  risk  management  goals  indicate  the  result  that  will  be  achieved  by  effec¬ 
tive  implementation  of  the  institutionalization  features  and  activities  of  a  key  process 
area.  The  goals  highlight  both  the  scope  and  the  intent  of  the  acquisition  risk  manage¬ 
ment  key  process  area. 
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Section  1.1 
Goall 


Project-wide  participation  in  the  identification  and  mitigation  of  risks  is  encour¬ 
aged  and  rewarded. 

Often,  project  management  is  not  proactive  about  identifying  and  addressing  potential 
problems.  Consequently,  managers  do  not  encourage  project  members  to  identify  and 
mitigate  risks.  The  assumption  is  that  everything  will  progress  according  to  plan,  and 
there  are  no  contingency  plans  available  when  things  do  go  wrong  [Charette  89].  Per¬ 
sonnel  on  projects  where  risk  is  systematically  managed  do  identify  risks  which  could 
jeopardize  those  projects.  The  risks  can  then  be  proactively  addressed  and  mitigated. 
Managers  must  deal  with  risk  information  in  a  positive  manner  and  reward  those  who 
identify  risks  to  sustain  and  reinforce  this  type  of  behavior  in  their  organizations  [Doro- 
fee  96].  This  is  done  by  establishing  a  culture  where  risks  are  identified  and  addressed 
as  a  part  of  everyday  business. 

Effective  risk  management  requires  a  systematic  process  for  managing  potential  prob¬ 
lems,  domain  experience  and  expertise  of  the  project  personnel,  a  repository  of  risk  data, 
and  a  risk-aware  culture.  The  objective  of  Goal  1  is  to  establish  a  culture  in  which  risk 
information  is  openly  shared  and  where  risks  are  proactively  addressed.  This  can 
involve  modifying  an  organization’s  present  culture  to  create  and  sustain  an  environ¬ 
ment  that  enhances  risk  communication  and  removes  the  barriers  to  it. 

Risk  communication  deals  with  uncertainty  and  negative  consequences,  which  are  two 
subjects  that  most  people  have  difficulty  discussing.  Communication  is  essential  for 
managing  risks  within  an  organization.  Risk  communication  must  allow  a  free  flow  of 
information  within  and  between  all  project  levels,  value  the  individual  voice,  and  pre¬ 
serve  non-attribution  and  trusted  use  of  data.  If  done  successfully,  it  will  surface  rele¬ 
vant  issues  and  potential  problems  on  a  project,  and  as  a  result,  project  personnel  will 
feel  that  they  are  informed  [NRC  89]. 

Management  is  instrumental  in  establishing  and  sustaining  an  environment  that  encour¬ 
ages  risk  communication.  The  following  list  includes  a  few  of  the  environmental  and 
cultural  traits  that  can  help  to  enhance  risk  communication  in  an  organization: 

•  establishing  upper  management  sponsorship  of  risk  management 

•  rewarding  positive  behavior 

•  making  risk  actions  and  decisions  visible  to  project  members 

•  setting  an  example  by  being  a  role  model 

•  selecting  a  risk  management  advocate  within  the  organization  to  help  sustain  the 
motivation  for  risk  management 

While  management  must  establish  an  environment  that  enhances  risk  communication,  it 
must  also  work  to  remove  the  barriers  that  discourage  risk  communication.  The  follow¬ 
ing  list  includes  a  few  of  the  environmental  and  cultural  traits  that  can  help  to  inhibit  risk 
communication  in  an  organization: 

•  failing  to  establish  upper  management  sponsorship  of  risk  management 

•  providing  a  solution  before  a  problem  is  understood 

•  blaming  project  members  who  identify  issues  or  problems 

•  executing  “hidden  agendas” 

•  lacking  trust  in  other  project  members 
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Project 

Functions 


Managing  Risk 
in  the  Acquisi¬ 
tion  Process 


The  project  team’s  defined  software  acquisition  process  provides  for  the  identihea- 
tion,  an^ysis,  and  mitigation  of  risks  for  all  project  functions. 

In  the  defined  maturity  level  of  the  SA-CMM  (Level  3),  the  acquisition  organization’s 
software  acquisition  process  is  standardized.  The  process  is  then  tailored  and  defined 
for  each  project  in  the  organization.  At  Level  3,  both  project  management  and  contract 
management  activities  are  proactive  in  nature  [Fisher  99];  the  objective  is  to  address 
issues  before  they  become  problems.  Risk  management  is  a  way  to  identify  potential 
problems  and  take  action  to  prevent  the  problems  from  occurring.  It  is  a  proactive  pro¬ 
cess  that  is  integral  to  a  project’s  defined  software  acquisition  process. 

The  objective  of  Goal  2  is  for  project  personnel  to  proactively  manage  risk  as  part  of  the 
project’s  defined  software  acquisition  process  and  that  all  project  functions  are  consid¬ 
ered  as  sources  of  risk.  Effective  risk  management  requires  a  systematic  process  for 
managing  risk,  domain  experience  and  expertise  of  the  project  personnel,  a  repository  of 
risk  data,  and  a  risk-aware  culture.  When  effective  risk  management  is  employed  on  a 
project,  management  of  the  project  becomes  proactive,  and  potential  problems  are  iden¬ 
tified  and  addressed  early  [Charette  89]. 

On  many  projects,  team  members  consider  the  technical  aspects  of  the  system  when 
identifying  risk  areas  and  largely  ignore  non-technical  aspects.  Experts  estimate  that 
90%  of  the  technical  problems  seen  on  a  project  are  attributable  to  problems  in  the  pro¬ 
cess  the  project  chooses  to  use  and  external  constraints  placed  on  the  project  [DoD  87]. 
Using  a  structured  list  of  interview  questions  [Carr  93]  that  span  product,  process,  and 
constraint  areas,  a  project  team  can  ensure  all  aspects  of  the  project  are  considered  when 
identifying,  and  thus  mitigating,  risks. 

Risk  must  be  managed  from  the  earliest  phases  of  an  acquisition  until  the  acquisition  has 
been  completed.  This  includes  all  pre-development  activities,  development  activities, 
and  post-development  activities  of  a  software  acquisition.  The  project  team  should 
incorporate  risk  when  developing  the  program  plan,  the  acquisition  strategy,  the  solicita¬ 
tion,  and  the  source  selection  plan  as  well  as  when  evaluating  proposals  and  selecting  a 
developer.  After  a  contractor  has  been  selected,  the  project  team  will  normally  manage 
the  high-level  or  project  risks,  while  the  developer  will  manage  the  risks  related  to  prod¬ 
uct  development  and  the  development  process.  In  many  cases,  the  project  team  and  its 
developers  can  work  together  to  cooperatively  manage  risk;  this  can  be  the  most  effec¬ 
tive  way  to  manage  risk  on  a  project  [Gluch  95].  Finally,  during  post-development 
activities,  such  as  transitioning  a  system  to  maintenance  and  support,  the  project  team 
must  also  be  sure  to  continuously  manage  risk  while  performing  its  tasks. 
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Project  reviews  include  the  status  of  identified  risks. 


Risk  management  is  often  seen  as  a  side  activity  on  projects.  When  “the  last  Tuesday  in 
the  month”  is  set  aside  as  “Risk  Day  ”  the  project’s  culture  naturally  conveys  that  there 
is  “real  work”  to  do  in  managing  a  project,  and  then  there’s  the  “risk  stuff.” 

For  those  organizations  where  the  stakes  are  high  (meaning  program  delivery  is  highly 
visible,  political,  or  innovative)  and  the  margin  for  error  is  low  (e.g.,  financial  margins, 
hard  delivery  dates,  tolerance  for  replanning),  decision  makers  need  to  be  able  to  make 
decisions  based  on  all  available  information.  Risk  information  must  be  reviewed  when 
making  decisions  that  effect  the  project’s  technical,  schedule,  or  funding  baselines. 

To  accomplish  reviews  with  efficiency  and  broad  system  understanding,  projects  need  to 
select  data  representation  formats  that  help  capture,  present,  and  track  risk  information 
in  a  clear  and  concise  manner.  See  Appendix  C,  p.  79  for  more  information  on  methods 
and  tools  to  use  when  tracking  and  presenting  risks  during  status  reviews. 
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Activities  Performed 

The  acquisition  risk  management  activities  are  the  steps  taken  or  functions  performed, 
either  mental  or  physical,  toward  achieving  the  KPA  goals.  Activities  include  all  of  the 
work  that  the  managers  and  technical  staff  do  to  perform  the  tasks  of  the  project  or  orga¬ 
nization. 
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Software  acquisition  risk  management  activities  are  integrated  into  software 
acquisition  planning. 

The  risk-related  information  generated  by  a  project’s  risk  management  process  is  used 
when  developing  software  acquisition  strategies  and  plans.  Software  acquisition  plan¬ 
ning  involves  preparing  the  strategies  and  plans  for  the  software-related  areas  of  an 
acquisition  project.  After  risks  are  identified,  the  project  team  can  proactively  incorpo¬ 
rate  the  risk  mitigation  plans  into  its  strategies  and  plans  and  can  address  the  risks  long 
before  they  become  problems. 

The  objective  of  Activity  1  is  for  project  teams  to  consider  risk  information  when  devel¬ 
oping  software  acquisition  strategies  and  plans.  Effective  risk  management  enables  pro¬ 
active  project  management,  allowing  a  project  team  to  identify  and  address  potential 
problems  early  in  the  acquisition  process.  Potential  problems  can  be  addressed  while  a 
project  team  is  formulating  its  acquisition  strategy  and  generating  its  acquisition  plan. 

Software  acquisition  planning  involves  preparing  the  strategies  and  plans  for  the  soft- 
ware-related  areas  in  system-level  planning  (e.g.,  budgetary  action,  schedule  determina¬ 
tion,  acquisition  strategy,  software  requirements  definition,  and  risk  identification, 
analysis  and  mitigation  planning)  [Fisher  99].  It  ensures  that  a  reasonable  planning 
effort  is  performed  for  the  software  acquisition  and  that  all  elements  of  the  project  are 
considered  when  developing  the  plan.  All  planning  activities  are  performed  and  docu¬ 
mented,  and  participation  in  system-level  planning  activities  is  included  as  appropriate. 
Software  acquisition  planning  begins  when  reasonable  resources  are  assigned  to  form  an 
acquisition  project  team,  independent  of  whether  the  team  is  formally  established  as  an 
organizational  entity. 

The  SEI  risk  management  paradigm  is  a  process  that  can  be  used  to  manage  risk  on  an 
acquisition  project.  It  consists  of  the  following  functions:  identify,  analyze,  plan,  track, 
control,  and  communicate  (see  Appendix  C,  p.  79,  for  methods  and  tools).  When  a 
project  team  performs  risk  identification,  analysis,  and  planning  early  in  a  project’s  life 
cycle,  it  can  use  the  information  as  it  develops  the  software  acquisition  strategy.  After 
the  need  for  the  system  has  been  established,  the  project  team,  users,  and  other  appropri¬ 
ate  personnel  can  identify  risks,  analyze  and  prioritize  the  risks,  and  develop  mitigation 
plans  for  the  most  important  risks.  Tlie  project  team  can  then  incorporate  the  mitigation 
plans  into  its  acquisition  strategy  and  plan. 

Note:  Communication  is  a  vital  part  of  the  risk  management  paradigm.  Most  of  the 
methods  and  tools  used  in  risk  management  require  communication  among  project  team 
members,  users,  and  other  personnel  involved  in  the  process. 

A  project  team  intends  to  acquire  a  system  which  pushes  the  envelope  of  current  techni¬ 
cal  knowledge.  A  risk  concerning  the  lack  of  foundational  work  in  the  technical  area 
was  identified.  The  mitigation  plan  for  this  risk  calls  for  an  incremental  development 
approach,  where  the  first  stage  is  the  development  of  a  rapid  prototype  for  proof  of  con¬ 
cept.  The  acquisition  strategy  is  modified  to  the  incremental  approach  because  of  the 
assessment  that  conventional  approaches  might  fail. 
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Activity  1  incorporates  risk  identification,  analysis,  and  planning  into  a  project’s  soft¬ 
ware  acquisition  strategy  and  plan.  Identification  produces  tangible  risk  statements  as 
well  as  supporting  context  information.  Everyone  on  a  project  is  responsible  for  identi¬ 
fying  risks.  Analysis  produces  a  prioritized  list  of  the  project’s  risks  (i.e.,  a  top  N  list). 
Personnel  who  have  the  appropriate  knowledge,  expertise,  and  background  to  effec¬ 
tively  deal  with  risk  information  are  responsible  for  evaluating,  classifying,  and  priori¬ 
tizing  the  risks.  Planning  produces  appropriate  plans  (i.e.,  research  plan,  acceptance 
rationale,  tracking  requirements,  or  mitigation  plan)  for  the  most  important  risks  (i.e., 
the  top  N  risks).  Personnel  who  have  the  knowledge,  expertise,  background,  and 
resources  to  effectively  deal  with  risks  are  responsible  for  developing  their  plans. 

Note:  Detailed  information  about  risk  identification,  analysis,  and  planning  can  be 
found  in  Chapter  1  p.  1,  and  information  about  the  methods  and  tools  that  support  iden¬ 
tification,  andysis,  and  planning  can  be  found  in  Appendix  C,  p.  79. 

Baseline  risk  identification  and  analysis  is  a  process  that  establishes  a  baseline  set  of 
risks  early  in  a  project.  It  is  a  concentrated  effort  using  many  of  the  identification  and 
analysis  tools  listed  in  Appendix  C,  p.  79,  to  capture  and  assess  all  of  the  risks  that  are 
presently  known  by  project  personnel.  The  selection  of  methods  and  tools  used  during 
this  process  is  driven  by  the  project’s  needs  and  goals.  The  output  of  baseline  risk  iden¬ 
tification  and  analysis  is  multiple  sets  of  related  risks  (also  referred  to  as  risk  areas  or 
mitigation  areas).  Baseline  risk  planning  typically  follows  baseline  risk  identification 
and  analysis. 

Baseline  risk  planning  is  a  process  that  develops  integrated  mitigation  plans  for  the  mul¬ 
tiple  sets  of  related  risks  that  are  captured  during  baseline  risk  identification  and  analy¬ 
sis.  It  is  a  concentrated  effort  using  many  of  the  planning  tools  listed  in  Appendix  C,  p. 
79,  to  develop  mitigation  plans  for  the  most  important  risk  sets.  The  priority  of  sets  and 
individual  risks,  as  determined  by  the  project  team,  drives  how  much  planning  is  done. 
Risks  and  risk  sets  that  are  not  considered  to  be  a  priority  are  either  accepted  or  watched. 
The  selection  of  methods  and  tools  used  during  this  process  is  driven  by  the  project’s 
needs  and  goals. 

Note:  It  is  important  to  build  baseline  mitigation  plans  as  soon  as  possible  after  perform¬ 
ing  baseline  identification  and  analysis. 

Performing  baseline  identification  and  analysis  and  baseline  planning  early  in  a  project’s 
life  cycle  can  help  the  project  team  to  develop  its  software  acquisition  strategy.  After 
the  need  for  the  system  has  been  established,  a  baseline  can  be  generated  with  participa¬ 
tion  from  the  project  team,  users,  and  other  appropriate  personnel.  The  output  of  this 
process  will  be  integrated  mitigation  plans  for  the  highest  priority  risks  and  risk  sets. 
The  project  team  can  then  incorporate  the  mitigation  plans  into  the  project’s  acquisition 
strategy  and  plans. 

•  Project  teams  should  consider  risk  information  when  developing  software  acquisition 
strategies  and  plans. 

•  The  output  of  risk  identification  should  be  tangible  risk  statements  and  supporting 
context  information. 

•  Everyone  on  a  project  should  be  responsible  for  identifying  risks. 

•  The  output  of  risk  analysis  should  be  a  prioritized  list  of  the  project’s  risks  (i.e.,  a  top 
Nlist). 
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•  Personnel  who  have  the  appropriate  knowledge,  expertise,  and  background  to 
effectively  deal  with  risk  information  should  be  responsible  for  evaluating,  classifying, 
and  prioritizing  the  risks. 

•  The  output  of  risk  planning  should  be  appropriate  plans  (i.e.,  research  plan,  acceptance 
rationale,  tracking  requirements,  or  mitigation  plan)  for  the  most  important  risks  (i.e., 
the  top  N  risks). 

•  Personnel  who  have  the  knowledge,  expertise,  background,  and  resources  to 
effectively  deal  with  risks  should  be  responsible  for  developing  their  plans. 

•  Risk  identification,  analysis,  and  planning  should  be  performed  early  in  a  project’s  life 
cycle  to  help  the  project  team  to  develop  its  software  acquisition  strategy. 
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The  Software  Acquisition  Risk  Management  Plan  is  developed  in  accordance  with 
the  project’s  defined  software  acquisition  process. 

At  the  Defined  Level  of  the  SA-CMM  (Level  3),  the  acquisition  organization’s  standard 
software  acquisition  process  is  integrated  into  each  project.  The  project’s  defined  pro¬ 
cess  is  tailored  from  the  acquisition  organization’s  process,  addressing  specific  charac¬ 
teristics  of  the  project  [Fisher  99].  The  management  plans  for  a  project  are  based  on  the 
project’s  defined  acquisition  process,  and  the  risk  management  plan  is  one  of  the 
project’s  management  plans.  A  project’s  Software  Acquisition  Risk  Management  Plan 
(also  referred  to  as  the  risk  management  plan)  documents  how  risks  will  be  managed  on 
the  project.  It  includes  the  processes,  activities,  milestones,  and  responsibilities  associ¬ 
ated  with  risk  management. 

The  objective  of  Activity  2  is  for  a  project  team  to  define  how  the  risk  management  pro¬ 
cess  (see  Chapter  1,  p.  1)  will  be  applied  to  the  project.  This  is  achieved  by  developing 
the  Software  Acquisition  Risk  Management  Plan  [Charette  89],  which  is  based  on  die 
project’s  defined  acquisition  process.  The  risk  management  plan  documents  the  process 
that  will  be  used  to  identify  and  address  potential  problems  early  in  the  acquisition. 

The  Software  Acquisition  Risk  Management  Plan  should  be  tailored  for  the  particular 
processes,  methods,  and  tools  used  by  the  project  team.  Managers  have  latitude  to  struc¬ 
ture  the  document  to  suit  their  needs  [DSMC  89].  The  following  table  lists  the  minimal 
recommended  content  for  a  risk  management  plan  [Dorofee  96]: 

Part  Description 

Introduction  The  introduction  defines  the  purpose  and  scope  of  the  plan 

as  well  as  the  content  that  can  be  found  in  the  Software 
Acquisition  Risk  Management  Plan.  Any  assumptions, 
constraints,  and  policies  for  implementing  the  processes  as 
well  as  any  related  plans,  documents,  and  standards  are  also 
found  here. 

Overview  of  processes  The  overview  of  processes  describes  the  risk  management 
activities  and  how  they  are  related  to  each  other;  provides  all 
process  and  data  flows;  and  describes  how  the  risk 
management  activities  are  integrated  with  other  project 
management  activities. 
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Part  Description 

Organization  The  organization  section  of  the  Software  Acquisition  Risk 

Management  Plan  includes  information  about 

•  project  organization  and  responsibilities 

•  customer  responsibilities 

•  supplier  responsibilities 

•  co-developer  responsibilities 

This  information  includes:  a  description  of  the  project 
organization;  an  organization  chart  that  maps  risk 
management  activities  to  project  roles  and  management 
responsibilities  (see  Commitment  2,  p.  46);  and  a  list  of  the 
risk  management  responsibilities,  activities,  and  products 
expected  from  customers,  suppliers,  and  co-developers. 

Process  details  The  process  details  describe  the  processes  and  procedures 

required  for  systematic  risk  management  (see  Chapter  1, 
p.  1,  Activity  1,  p.  25,  and  Activity  6,  p.  37).  This  part  of 
the  Software  Acquisition  Risk  Management  Plan  also 
includes 

•  the  methods  and  tools  that  are  chosen  to  support  each 
function  as  well  as  the  criteria  for  selecting  one  method  or 
tool  over  another 

•  references  to  other  plans,  handbooks,  and  training 
materials  for  any  method  or  tool  that  is  documented 
elsewhere  (e.g.,  in  the  project’s,  organization’s,  or 
customer’s  related  materi^s) 

•  all  process  improvement  metrics  that  must  be  collected 
and  reported  (e.g.,  the  number  of  risks  open,  the  risks’ 
classifications,  the  number  of  successful  mitigations,  the 
number  of  failed  mitigations,  etc.,  see  Measurement  1, 
p.51) 

•  the  process  required  to  evaluate  and  improve  the  risk 
management  process  (e.g.,  a  quarterly  evaluation  of  the 
methods  for  their  efficiency,  a  periodic  review  of 
customer  reports  assessing  Aeir  usefulness,  etc.) 

Resources  and  The  resources  and  schedule  section  documents  the 

schedule  resources  (e.g.,  cost,  staff  effort,  equipment,  software) 

required  for  the  risk  management  process.  The  allocated 
budget  as  well  as  the  source  of  mitigation  funds  are  also 
specified  (see  Ability  2,  p.  48).  A  mapping  of  risk  manage¬ 
ment  activities  against  the  project  schedule  and  milestones 
is  included  in  this  section  of  the  risk  management  plan.  All 
risk  management-related  deliverables,  such  as  risk  sum¬ 
mary  reports,  baseline  results,  and  mitigation  plans,  are  also 
documented  here. 
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Current  List  of 
Risks  and  Miti¬ 
gation  Plans 


Tailoring  a  Risk 

Management 

Plan 


Recommenda¬ 
tions  for 
Activity  2 


Part  Description 

Risk  documentation  Risk  documentation  defines  the  database  tool  specifica¬ 

tions,  including  access  to,  control  of,  and  management  of 
databases.  Any  templates  or  forms  that  are  required  should 
be  either  included  in  this  part  of  the  plan  or  referenced 
appropriately.  All  procedures  and  requirements  for  com¬ 
pleting,  processing,  controlling,  and  retaining  risk-related 
documents  and  forms  should  also  be  provided  here. 


The  current  list  of  risks  and  their  mitigation  plans  can  be  included  in  the  Software 
Acquisition  Risk  Management  Plan.  If  the  project  team  continuously  manages  acquisi¬ 
tion  risks,  then  it  could  be  faced  with  an  administrative  burden  as  it  continually  updates 
the  risk  management  plan  to  reflect  changes  to  the  list.  It  is  recommended  that  the  cur¬ 
rent  list  of  risks  and  their  mitigation  plans  be  maintained  and  updated  separately  from 
the  risk  management  plan. 

The  method  used  by  a  project  team  to  document  the  Software  Acquisition  Risk  Manage¬ 
ment  Plan  is  ultimately  determined  by  the  project  team  and  the  acquisition  organization. 
The  risk  management  plan  should  be  tailored  for  the  processes,  methods,  and  tools  used 
by  the  project  team;  it  can  be  part  of  the  system-level  risk  management  plan,  part  of  the 
Project  Management  Plan  (see  the  Project  Performance  Management  KFA  of  the  SA- 
CMM),  or  a  stand-alone  plan.  The  factors  that  can  affect  how  a  project  team  decides  to 
construct  the  risk  management  plan  include:  the  size  of  the  project,  how  the  acquisition 
organization  does  business,  the  complexity  of  the  project,  and  the  composition  and  size 
of  the  project  team.  The  Software  Acquisition  Risk  Management  Plan  does  not  have  to 
be  any  more  extensive  than  the  risk  management  plans  developed  by  well-managed  soft¬ 
ware  acquisition  projects  [Fisher  99]. 

•  A  project  team’s  Software  Acquisition  Risk  Management  Plan  should  define  how  the 
risk  management  process  will  be  applied  to  the  project. 

•  The  Software  Acquisition  Risk  Management  Plan  should  be  based  on  the  project’s 
defined  acquisition  process. 

•  The  Software  Acquisition  Risk  Management  Plan  should  be  tailored  for  the  particular 
processes,  methods,  and  tools  used  by  the  project  team;  it  can  be  part  of  the  system- 
level  risk  management  plan,  part  of  die  Project  Management  Plan,  or  a  stand-alone 
plan. 

•  The  minimal  content  for  a  risk  management  plan  should  include 

•  introduction-defines  the  purpose  and  scope  of  the  risk  management  plan 

•  overview  of  processes-describes  all  risk  management  activities  and  their  relations  to 
other  project  management  activities  (see  Activity  1,  p.  25,  and  Activity  6,  p.  37) 

•  organization-defines  project  personnel  responsibilities  (see  Commitment  2,  p.  46)  as 
well  as  customer,  supplier,  and  co-developer  responsibilities 

•  process  details-describes  the  processes  and  procedures  for  systematic  risk 
management 

•  resources  and  schedule-documents  the  resources  (e.g.,  cost,  staff  effort,  equipment, 
software)  required  for  the  risk  management  process  (see  Ability  2,  p.  48) 

•  risk  documentation-defines  project  templates  and  forms,  database  tool 
specifications,  and  procedures  and  requirements  for  documentation 

•  The  current  list  of  risks  and  their  mitigation  plans  should  be  maintained  and  updated 
separately  from  the  risk  management  plan. 
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Modifying  a  Risk 
Management 
Plan 


Recommenda¬ 
tions  for 
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Activity  3 


The  project  team  performs  its  software  acquisition  risk  management  activities  in 
accordance  with  its  documented  plans. 

The  management  plans  for  a  project  are  based  on  the  project’s  defined  acquisition  pro¬ 
cess.  The  Software  Acquisition  Risk  Management  Plan,  which  is  one  of  the  project’s 
management  plans,  can  be  part  of  the  system-level  risk  management  plan,  part  of  the 
Project  Management  Plan,  or  a  stand-alone  plan.  The  format  of  the  plan  is  determined 
by  the  project  team  cind  the  acquisition  organization.  Once  the  Software  Acquisition 
Risk  Management  Plan  has  been  formally  documented,  project  personnel  must  perform 
their  risk  management  activities  (e.g.,  identifying  risks,  analyzing  risks)  as  described  in 
the  plan. 

The  objective  of  Activity  3  is  for  a  project  team  to  follow  the  Software  Acquisition  Risk 
Management  Plan.  Following  the  risk  management  plan  is  important  because  it  enables 
project  personnel  who  are  responsible  for  a  task  or  activity  to  perform  it  in  a  repeatable 
way.  Following  the  plan  also  helps  other  personnel  who  have  general  knowledge  of  the 
area  to  learn  and  perform  the  task  or  activity  as  outlined  in  the  plan.  In  addition,  people 
who  depend  on  the  consistency  of  the  results  can  be  satisfied.  Iliis  is  one  aspect  of  insti¬ 
tutionalizing  a  process  [Paulk  95]. 

A  project  team  has  developed  a  Software  Acquisition  Risk  Management  Plan  and  is  fol¬ 
lowing  the  plan  as  it  manages  project  risks,  ^oject  team  members  learn  about  a  new 
risk  tracking  tool  that  they  would  like  to  incorporate  into  the  team’s  risk  management 
process.  They  modify  their  risk  management  plan  to  reflect  the  use  of  the  new  tracking 
tool,  and  they  also  include  the  criteria  that  were  used  to  select  the  tool  in  the  risk  man¬ 
agement  plan. 

•  The  project  team  should  follow  the  Software  Acquisition  Risk  Management  Plan. 
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Activity  4 


Activity  4 


The  project  team  encourages  and  rewards  project-wide  participation  in  the  identi- 
Hcation  and  mitigation  of  risks. 


Description 


Objective 


Changing  the 
Culture 


Strong 

Sponsorship 


Example 


Recommenda¬ 
tions  for 
Activity  4 


The  people  who  know  most  about  the  risks  on  a  project  are  the  project  team  members 
and  affected  groups.  However,  studies  have  shown  that  on  most  failed  software-inten¬ 
sive  system  acquisitions,  the  team  members  didn’t  have  the  opportunity  to  voice  their 
concern  about  potential  problems  [Glass  98]. 

The  objective  of  Activity  4  is  to  create  an  environment  where  risks  are  identified  and 
mitigated  by  all  project  members.  A  “risk-aware”  culture  rewards  team  members  who 
identify — and  work  to  mitigate — disasters. 

For  most  organizations,  effective  risk  management  requires  some  culture  change.  One 
reason  why  risk  management  is  not  naturally  and  widely  practiced  is  because  it  chal¬ 
lenges  widespread  cultural  assumptions  in  most  U.S.  organizations.  Risk  management 
requires  identification  of  “bad  news  that  hasn’t  happened  yet”  in  an  environment 
(project  management)  that  seeks  the  positive  and  focuses  on  the  present,  not  the  future. 

Leaders  send  messages  whether  they  wish  to  or  not.  Project  team  members  are  con¬ 
stantly  looking  to  their  leaders  for  cues  about  what  is  acceptable  behavior.  It’s  not  just 
public  statements  that  are  heard  and  believed;  it’s  the  entire  range  of  messages  sent 
through  behaviors  and  their  consequences.  The  activities  that  leaders  pay  attention  to, 
measure,  and  make  decisions  about  are  the  activities  team  members  pay  attention  to  and 

perform.^ 

On  one  project,  the  team  members  were  concerned  that  if  they  instituted  a  risk  identifi¬ 
cation  process  that  was  too  formal,  project  members  would  feel  intimidated  about  bring¬ 
ing  forward  potential  problems.  Sensitive  to  the  concerns,  the  project  manager  decided 
to  have  informal  risk  identification  sessions  over  coffee  on  a  weekly  basis.  All  members 
were  invited  to  participate  and  one  person  was  tasked  with  taking  notes.  The  weekly 
meeting,  called  the  “Starbucks  Risk  Roundup,”  allowed  all  team  members  to  discuss 
risk  information  in  a  non-threatening  environment. 

•  Project  managers  should  pay  attention  to,  measure,  and  control  desired  risk  practices, 
activities,  and  behaviors. 

•  Teach  and  use  the  Condition  ->  Consequence  [Dorofee  96]  form  for  capturing  concerns 
and  issues  (potential  risks)  in  the  organization. 

•  Encourage  project  team  members  to  raise  concerns  and  issues  without  requiring  that 
they  have  a  solution  in  hand  for  them — ^but  make  sure  the  concerns  or  issues  are 
documented  in  the  project  risk  database  once  they  are  raised. 

•  Make  it  easy  to  document  an  issue,  but  hard  to  make  it  go  away;  reward  risk 
identification,  not  risk  closure. 


1 .  Loveland  Link,  Jo  Lee,  et  al.  Rollout  and  Installation  of  Risk  Management  at  the 
MINT  Directorate,  National  Reconnaissance  Office  (CMU/SEI-99-TR-009).  Pitts¬ 
burgh,  Pa.:  Software  Engineering  Institute,  Carnegie  Mellon  University,  1999. 
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Solicitation 


Solicitation 

Example 


Project 

Performance 

Management 


Contract 

Performance 

Management 


Risk  management  is  conducted  as  an  integral  part  of  the  solicitation,  project  per¬ 
formance  management,  and  contract  performance  management  processes. 


Software  acquisition  risk  management  is  performed  as  an  integral  part  of  the  project 
team’s  activities.  The  project  team  considers  data  collected  from  software  acquisition 
risk  management  activities  when  making  decisions.  This  focus  on  “what  could  go 
wrong”  is  the  major  factor  in  helping  a  project  team  shift  from  reacting  to  problems  as 
they  arise  to  anticipating  and  avoiding  problems. 

The  objective  of  Activity  5  is  to  ensure  that  software  acquisition  risk  management  is 
integrated  into  the  way  a  project  team  manages  the  project.  Risk  management  isn’t  just 
conducted  during  “risk  week”  or  during  a  concentrated  effort  to  write  a  risk  manage¬ 
ment  plan.  Software  acquisition  risk  management  has  a  role  in  all  activities  of  the 
project  team. 

During  solicitation,  the  project  team  prepares  a  solicitation  package  and  selects  a  con¬ 
tractor  who  is  best  capable  of  satisfying  the  requirements  of  the  contract.  The  contract 
type  (e.g.,  fixed-price,  cost-reimbursement)  and  the  acquisition  approach  (e.g.,  incre¬ 
mental  acquisition,  prototyping)  should  be  chosen  based  on  the  risk  of  not  meeting  the 
requirements  of  the  acquisition  within  cost  and  schedule  constraints.  The  project  team 
may  consider  asking  the  contractor  to  submit  a  software  risk  management  plan  in 
response  to  the  solicitation  [Fisher  99]. 

A  project  team  is  acquiring  software  to  control  a  robot  that  will  be  used  to  explore  haz¬ 
ardous  terrain.  Based  on  the  operational  requirements,  the  robot  must  be  autonomous 
rather  than  controlled  by  teleoperation  or  remote  control.  The  software  to  allow  a  robot 
to  operate  completely  autonomously  is  extremely  complicated  and  may  even  be  beyond 
current  technical  capability.  Given  the  uncertainty  of  die  technical  solution  and  the  risk 
of  failure,  the  project  team  selects  a  cost-plus  incentive  fee  contract  type  and  requires  an 
incremental  development  approach  as  a  part  of  their  risk  mitigation  strategy. 

At  Level  3  of  the  SA-CMM,  the  project  team  manages  the  software  acquisition  project 
according  to  a  defined  software  acquisition  process.  The  Project  Management  Plan 
addresses  all  of  the  project  team’s  management  planning,  including  risk  management 
planning  (see  Activity  2,  p.  28,  for  a  discussion  of  the  Software  Acquisition  Risk  Man¬ 
agement  Plan).  The  project  team  applies  a  systematic  approach  while  they  identity  and 
analyze  risks  and  plan  risk  handling  (risk  mitigation)  actions  (see  Activity  1,  p.  25  and 
Activity  6,  p.  37)  [Fisher  99]. 

During  contract  performance  management,  the  project  team  uses  a  defined  contract 
management  process  to  ensure  the  acquired  software  products  and  services  satisfy  con¬ 
tract  requirements.  Risk  analysis  and  management  is  performed  by  the  project  team  as 
an  integral  part  of  contract  performance  management.  The  project  team  follows  its 
plans,  which  include  risk  management.  The  project  team  appraises  the  contractor’s  risk 
management  system  and  measures  the  risk  analysis  process  [Fisher  99].  The  project 
team  also  ensures  that  the  contractor  is  managing  risk  as  outlined  in  the  contract  as  well 
as  in  the  contractor’s  risk  management  plan. 


Managing  Risk  Contractors  play  an  important  role  in  managing  software  acquisition  risks.  Project 

with  Contractors  teams  can’t  simply  transfer  risk  to  the  contractor  after  contract  award.  The  project  team 
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is  ultimately  responsible  for  the  success  of  the  project  and  is  primarily  responsible  to 
ensure  that  risks  are  identified  and  mitigated.  The  project  team  and  contractor(s)  should 
enter  into  a  teaming  relationship  where  risks  are  identified,  analyzed,  planned,  tracked, 
controlled,  and  communicated  in  a  shared  environment  (see  Section  4,  Managing  Risk 
with  Others,  p.  55).  The  effectiveness  of  joint  mitigation  strategies  is  greater  than  the 
sum  of  individual,  potentially  diverse,  contractor  and  project  team  risk  handling  plans. 

Risk  Manage-  Even  though  acquisition  risk  management  isn’t  defined  as  a  KPA  until  Level  3  of  the 

ment  at  Level  2  SA-CMM,  project  teams  begin  to  perform  basic  risk  management  as  an  integral  part  of 

Level  2  activities.  The  following  chart  shows  where  risk  management  is  performed  at 
Level  2. 


Level  2  KPA 

Key  Practice 

Risk-Related  Function 

Software  Acquisition 
Planning 

Commitment  1 

Written  policy  typically  includes  a  review  process  for 
resolving  issues  that  focus  on  critical  areas  such  as 
affordability  and  risk. 

Activity  3 

Risk  identification  is  a  part  of  the  software 
acquisition  strategy  development. 

Activity  4 

Risk  identification  and  tracking  is  addressed  in 
software  acquisition  planning. 

Solicitation 

Commitment  1 

Contract  type  is  chosen  based  on  perceived  risk. 

Activity  1 

The  project  team  may  decide  to  determine  an 
offeror’s  ability  to  perform  based  on  project  risk. 

Offeror  may  be  asked  to  submit  a  risk  management 
plan  and  perceived  risks  with  its  response  to  the 
solicitation. 

Proposal  evaluation  may  include  a  determination  of 
the  amount  of  risk  associated  with  a  given  offeror’s 
response  to  the  solicitation. 

Requirements 
Development  and 
Management 

Activity  4 

Changes  are  analyzed  for  risk. 

Project  Management 

Activity  1 

Risk  identification  and  tracking  are  performed 
according  to  plans. 

Activity  4 

The  project  team  tracks  the  risks  associated  with  cost, 
schedule,  resources,  and  the  technical  aspects  of  the 
project. 

Activity  7 

Plans  are  kept  current  as  appropriate  when  new  risks 
are  discovered. 

Contract  Tracking  and 
Oversight 

Activity  2 

Contractor’s  Software  Acquisition  Risk  Management 
Plan  is  reviewed  if  applicable. 

Evaluation 

Commitment  1 

Acquired  software  products  and  services  are 
evaluated  with  intent  of  reducing  acquisition  risk. 
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Level  2  KPA  Key  Practice  Risk-Related  Function 

Activity  1  Plans  describe  the  risks  addressed  by  the  evaluation. 

Activity  5  Independent  evaluations  may  be  performed  to  further 

reduce  risk  of  failure. 


Risk  Manage-  Acquisition  risk  management  resides  at  Level  3  of  the  SA-CMM.  It’s  the  highdeverage 

ment  at  Level  3  KPA  that  helps  a  project  change  its  focus  from  being  reactionary  to  proactively  manag¬ 
ing  the  project.  The  following  table  identifies  all  areas  of  Level  3  where  risk  manage¬ 
ment  is  performed. 

Level  3  KPA  Key  Practice  Risk-Related  Function 

Project  Performance  Activity  2  The  Project  Management  Plan  addresses  risk 

Management  management  planning. 

Activity  10  The  project  team  identifies  and  analyzes  risks  and 

identifies  risk  mitigation  actions. 


Contract  Performance 
Management  (CPM) 


Acquisition  Risk 
Management  (ARM) 


Goal  1 

The  quality  of  contractor  team  process,  performance, 
products,  and  services  is  appraised  throughout  the 
contract’s  period  of  performance  to  identify  risks  and 
take  action  to  mitigate  those  risks  as  early  as  possible. 

Conunitment  1 

Risk  analysis  and  management  is  done  as  a  part  of 
CPM  activities. 

Activity  1 

CPM  plans  include  risk  management. 

Activity  2 

The  contractor’s  risk  management  system  is 
appraised. 

Activity  5 

The  project  team  may  propose  changes  to  the 
software  acquisition  approach  to  mitigate  risks. 

Activity  7 

To  help  foster  a  cooperative  and  productive 
environment  among  the  end  user,  project  team,  and 
the  contractor  team,  the  methods  of  identifying  and 
mitigating  risks  are  improved  as  appropriate. 

Measurement  1 

Risk  analysis  process  is  measured. 

Commitment  1 

Written  policy  for  software  ARM  exists. 

Commitment  2 

Responsibility  for  performing  ARM  activities  is 
designated. 

Ability  1 

A  group  exists  to  coordinate  ARM. 

Ability  2 

Adequate  resources  are  provided  to  do  ARM. 

Ability  3 

Individuals  doing  ARM  have  experience  or  training. 
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Risk  Manage¬ 
ment  at  Levels  4 
and  5 


Key  Practice 
Activity  1 


Risk-Related  Function 

Software  ARM  is  integrated  into  software  acquisition 
planning. 


Activity  2 

Software  ARM  Plan  is  developed. 

Activity  3 

ARM  Plan  is  followed. 

Activity  4 

Project  team  encourages  and  rewards  participation. 

Activity  5 

ARM  is  integrated  with  other  KPAs. 

Activity  6 

Risks  are  tracked  and  controlled  until  mitigated. 

Activity  7 

Risk  status  briefed  during  project  reviews. 

Measurement  1 

The  statuses  of  ARM  activities  and  products  are 
measured. 

Verification  1 

ARM  activities  are  reviewed  by  acquisition 
organization  management. 

Verification  2 

ARM  activities  are  reviewed  by  project  manager. 

At  the  higher  maturity  levels  of  the  SA-CMM,  the  project  team  sets  quantitative  quality 
objectives  for  processes,  products,  and  services.  The  acquisition  organization  is  focused 
on  continuous  process  improvement.  The  following  tables  show  how  the  project  team 
and  acquisition  organization  integrate  software  acquisition  risk  management  into  their 
activities. 


Level  4  KPA 

Quantitative 

Acquisition 

Management 


Key  Practice 
Activity  5 


Risk-Related  Function 

Risk  management  practices  are  tracked  using 
quantitative  methods. 


Level  5  KPA 

Acquisition 

Innovation 

Management 


Key  Practice  Risk-Related  Function 

Commitment  1  Policy  describes  how  new  techniques  and 

technologies  are  evaluated  for  risk. 


Recommenda¬ 
tions  for 
Activity  5 


The  project  team  should  integrate  acquisition  risk  management  into  all  of  its  activities. 
The  contract  type  should  be  chosen  based  on  perceived  risk. 

The  project  team  should  identify,  analyze,  plan,  track,  and  control  risks  during  Project 
Performance  Management  activities. 

The  project  team  should  identify  the  risks  associated  with  the  contractor’s  activities. 
The  project  team  and  contractor(s)  should  enter  into  a  teaming  relationship  where  risks 
are  identified,  analyzed,  planned,  tracked,  controlled,  and  communicated  in  a  shared 
environment. 
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Activity  6 
Description 


I 


Objective 


Risk 

TVacking  and 
Control 


Activity  6  ^ 

Software  acquisition  risks  are  analyzed,  tracked,  and  controUed  until  mitigated. 

Mitigation  plans  are  developed  for  the  most  important  risks  to  the  project.  The  project 
tpam  must  track  both  the  progress  of  a  plan  and  its  effectiveness  in  reducing  the  risk  to 
the  project.  As  project  personnel  track  data,  they  must  make  “control  decisions.”  For 
example,  if  the  mitigation  is  working  as  intended,  then  the  decision  would  be  to  continue 
with  the  mitigation  plan.  If  there  is  a  problem  with  the  mitigation  plan,  then  a  new  plan 
might  be  required  or  a  contingency  plan  might  be  implemented.  If  the  mitigation  plan  is 
judged  to  be  successful,  then  the  risk  can  be  closed.  In  general,  risk  tracking  and  control 
includes  tracking  the  status  of  mitigation  actions  against  the  mitigation  plan;  tracking 
the  effectiveness  of  the  mitigation  plan;  reporting  tracking  data  to  the  appropriate  deci¬ 
sion  makers;  replanning  a  mitigation  plan  or  invoking  a  contingency  plan  when  neces¬ 
sary;  and  periodically  reviewing  data  about  the  statuses  of  risks  and  their  plans. 

The  objective  of  Activity  6  is  for  project  teams  to  know  if  a  mitigation  plan  is  being  exe¬ 
cuted  as  it  was  designed  and  to  understand  whether  a  mitigation  plan  is  effectively 
reducing  risk  to  the  project.  By  gathering  and  examining  the  metrics  that  are  defined 
during  risk  planning,  project  personnel  can  determine  whether  a  plan  is  being  executed 
properly  and  reducing  risk.  They  can  then  take  appropriate  actions  based  on  the  analysis 
of  the  data.  This  helps  to  ensure  that  risk  mitigation  plans  effectively  reduce  risk  to  the 
acquisition. 

Activity  6  focuses  on  tracking  and  controlling  mitigation  plans  until  the  risks  are  miti¬ 
gated.  Tracking  produces  reports  (e.g.,  documents  or  presentations)  highlighting  the  rel¬ 
evant  tracking  data.  The  person(s)  who  is  assigned  responsibility  for  tracking  watched 
and  mitigated  risks  prepares  the  reports.  Control  produces  decisions  (i.e.,  replan,  close 
the  risk,  invoke  a  contingency  plan,  or  continue  tracking  and  executing  the  current  plan) 
which  are  then  implemented  by  project  personnel.  The  person  who  has  accountability 
for  a  risk  should  make  the  control  decision  for  that  risk. 

Note'.  Detailed  information  about  risk  tracking  and  control  can  be  found  in  Chapter  1  of 
this  guidebook  (see  Section  1.2,  Identify,  Analyze,  Plan,  Track,  Control,  and  Communi¬ 
cate,  p.  6)  and  information  about  the  methods  and  tools  that  support  tracking  and  con¬ 
trol  can  be  found  in  the  appendices  (see  Appendix  C,  p.  79). 


Risk  Reporting  Risk  reporting  should  be  combined  with  routine  project  management  activities  (e.g.,  as 
part  of  a  weekly  or  monthly  project  status  update).  Risk  data  provides  more  information 
that  decision  makers  can  use  when  making  decisions.  The  frequency  of  reporting  can 
depend  upon 

•  the  reporting  requirements  for  each  risk  or  risk  set  as  outlined  during  planning  (e.g., 
weekly  or  bi-weekly) 

•  the  manner  in  which  the  report  will  be  used 

Note'.  A  critical  event  or  condition  might  require  that  information  be  reported  to  a  deci¬ 
sion  maker  immediately  rather  than  waiting  for  the  next  reporting  period. 
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Control 

Decisions 


The  following  table  describes  the  control  decisions  that  can  be  made  for  risks. 


Tracking  and 
Control  vs. 
Project 
Management 


Tracking  and 
Control  Example 


Component 

Description 

Replan 

Replanning  is  required  when  analysis  of  the  data  shows  that 
the  mitigation  plan  is  not  working  and  a  contingency  plan  is 
not  available. 

Close  the  risk 

Closed  risks  are  not  tracked  because  the  risks  no  longer 
exist  or  are  not  cost  effective  to  track.  This  occurs  when 

•  The  probability  or  impact  have  been  reduced  below  a 
defined  threshold. 

•  The  risk  has  become  a  problem  and  now  tracked  as  such. 

Note'.  All  closed  risks  should  be  documented  along  with  the 
rationale  for  closure.  Closure  of  a  risk  requires  the  agree¬ 
ment  of  all  affected  parties. 

Invoke  a  contingency 
plan 

Contingency  plans  are  alternative  plans  developed  ahead  of 
time.  They  are  invoked  when  the  data  indicates  that  a  plan 
is  not  working.  Risks  as  well  as  their  mitigation  plans  con¬ 
tinue  to  be  tracked  after  the  contingency  plan  has  been  exe¬ 
cuted. 

Continue  tracking  and 
executing  the  current 
plan 

When  the  analysis  of  the  tracking  data  indicates  that  all  is 
going  as  expected,  the  decision  maker  can  decide  to 
continue  tracking  the  risk  or  mitigation  plan  as  before. 

Risk  tracking  and  control  should  be  closely  related  to  standard  project  management 
monitoring  techniques  used  by  the  acquisition  organization.  One  of  the  goals  of  risk 
tracking  and  control  is  to  provide  the  project  team  with  more  information  to  use  when 
making  project  decisions.  Risk  management  activities  should  be  integrated  and  coordi¬ 
nated  with  existing  project  management  activities  for  the  project  team  or  acquisition 
organization. 

A  project  team  is  acquiring  software  to  use  in  a  product  it  is  developing.  Completion  of 
the  product  might  be  delayed  because  of  the  large  number  of  change  requests  being  sub¬ 
mitted  by  the  marketing  group.  Each  change  request  requires  a  modification  to  the  con¬ 
tract  before  it  can  be  implemented,  and  this  process  is  causing  delays  in  processing  and 
completing  the  changes.  The  company  could  miss  its  window  of  opportunity  for  this 
product  if  the  release  date  is  delayed.  However,  it  is  also  important  for  them  to  incorpo¬ 
rate  the  changes  in  order  to  develop  a  product  that  appeals  to  the  marketplace. 

The  mitigation  plan  for  this  risk  calls  for  the  product  team  to  negotiate  a  contract  vehicle 
where  a  specified  amount  of  resources  will  be  set  aside  for  future  changes  anticipated 
during  the  remainder  of  the  project.  As  change  requests  are  submitted,  contract  modifi¬ 
cations  will  no  longer  be  required  and  resources  to  implement  the  changes  will  be  avail¬ 
able.  The  project  team  chooses  to  track  the  resources  remaining  and  the  rate  of  resource 
consumption  for  this  risk.  As  the  project  progresses,  an  analysis  using  the  rate  of 
resource  consumption  and  the  resources  remaining  indicates  that  the  resources  desig¬ 
nated  for  processing  and  implementing  changes  will  be  consumed  prior  to  the  end  of  the 
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Recommenda 
tions  for 
Activity  6 


project.  The  project  manager  decides  to  negotiate  another  contract  vehicle  to  set  aside 

additional  resources  for  future  changes. 

•  Project  teams  should  determine  whether  plans  are  being  executed  properly  and 
reducing  risk  by  gathering  and  examining  the  risk  metrics  and  that  are  defined  during 
risk  planning. 

•  The  output  of  risk  tracking  should  be  documents  or  presentations  highlighting  the 
relevant  tracking  data. 

•  Tracking  should  be  performed  by  the  person(s)  responsible  for  tracking  watched  and 
mitigated  risks. 

•  The  output  of  risk  control  should  be  decisions  (i.e.,  replan,  close  the  risk,  invoke  a 
contingency  plan,  or  continue  tracking  and  executing  the  current  plan)  which  are  then 
implemented  by  project  personnel. 

•  The  person  who  has  accountability  for  a  risk  should  make  the  control  decision  for  that 
risk. 

•  Risk  reporting  should  be  combined  with  routine  project  management  activities  (e.g.,  as 
part  of  a  weekly  or  monthly  project  status  update)  to  provide  the  project  team  with 
more  information  to  use  when  making  project  decisions. 
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Activity  7 
Description 

Objective 

Ground  Rules  for 

Productive 

Reviews 


Section  2.7 


Activity  7 


Project  reviews  include  the  status  of  identified  risks. 

Productive  reviews  increase  the  chance  of  having  a  successful  project.  Reviews  where 
risk  management  is  an  integral  part  of  the  meeting  allow  project  managers  and  team 
members  to  make  informed  decisions  about  the  status  of  the  project  and  the  mitigation 
methods  selected  to  reduce  the  impact  or  probability  of  future  problems. 

The  objective  of  Activity  7  is  to  integrate  risk  management  into  the  project’s  existing 
review  meetings.  When  risk  management  is  treated  separately,  project  team  members 
and  other  participants  tend  to  view  managing  risk  as  an  unnecessary  and  overburdened 
addition  to  their  “regular”  duties.  In  order  to  fully  communicate  project  status,  and  to 
make  decisions,  project  teams  need  to  constantly  keep  an  eye  on  project  goals  and 
potential  barriers  in  achieving  those  goals. 

On  many  acquisition  projects,  meetings  and  reviews  are  seen  as  painful  and  unproduc¬ 
tive.  One  reason  for  this  perception  is  the  lack  of  meeting  management  skills.  The  fol¬ 
lowing  table  describes  basic  ground  rules  for  productive  meetings  and  can  be  used  as  a 
starting  point  for  project  teams  when  developing  their  own  ground  rules  [Scholtes  88]. 


Ground  Rule 

Description 

Use  agendas 

Each  review  must  have  an  agenda.  The  agenda  should  be 
distributed  to  participants  in  advance  and  should  contain  the 
following  information: 

•  the  agenda  topics  (a  sentence  or  two  for  each  defining  the 
item  and  why  it  is  being  discussed) 

•  the  presenter  or  discussion  lead  (the  person  responsible 
for,  or  knowledgeable  about  the  item) 

•  a  time  guideline  (time  needed  in  minutes  to  discuss  the 
item) 

•  the  item  type  (e.g.,  discussion,  decision,  announcement) 

Have  a  facilitator 

The  facilitator  is  responsible  for  keeping  the  review 
focused  and  moving.  Sometimes  the  project  lead  fills  this 
role.  However,  because  the  facilitation  role  is  so  important 
to  the  success  of  the  review,  it  is  better  to  choose  a  facilita¬ 
tor  who  can  pay  more  attention  to  the  review  process  and 
less  to  the  review  content. 

Take  minutes 

Each  review  should  have  someone  designated  to  take  notes. 
Issues,  risks,  actions,  and  decisions  all  need  to  be  recorded 
for  future  reference. 

Draft  next  agenda 

At  the  end  of  the  review,  draft  a  rough  agenda  for  the  next 
review. 

Evaluate  the  review 

Reviews  should  be  continuously  improved.  While  the 
review  is  fresh  in  the  participants’  minds,  a  simple  “plus/ 
minus”  discussion  at  the  end  is  vital  to  the  productivity  of 
future  reviews. 
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Essential  Risk 
Agenda  Items 
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Ground  Rule  Description 


Adhere  to  the  “100-  Once  the  review  starts,  all  those  in  attendance  should  give  it 
mile”  rule  their  full  attention.  Beepers  and  cell  phones  should  be 

turned  off  and  disruptions  limited  to  those  that  are  so 
important  they  would  occur  if  the  review  were  100  miles 
away  from  the  office. 


In  addition  to  the  above  ground  rules,  participants  need  to  develop  and  agree  upon 
acceptable  behavior  for  review  participants.  Risk  management  can  only  be  effective 
when  the  project  team  creates  an  environment  that  encourages  free  flowing  information 
at  and  between  all  project  levels  and  enables  formal,  informal,  and  impromptu  commu¬ 
nication.  Some  examples  of  acceptable  participant  behavior  are  listed  below. 

•  Test  assumptions  and  inferences. 

•  Share  all  relevant  information. 

•  Focus  on  interests,  not  positions. 

•  Be  specific — ^use  examples. 

•  Agree  on  what  important  words  mean. 

•  Explain  the  reasons  behind  one’s  statements,  questions,  and  actions. 

•  Disagree  openly  with  any  member  of  the  group. 

•  Make  statements,  then  invite  questions  and  comments. 

With  basic  meeting  management  skills  in  place,  the  content  of  the  project  review  can  be 
addressed.  One  govemment/contractor  team  adds  the  following  items  to  its  review 


agenda.^ 

Agenda  Item 

Description 

Roundtable 

Open  discussion  about  new  or  complex  factors  affecting  the 
team  and  its  ability  to  be  successful  in  achieving  project 
goals.  Purpose  is  to  set  the  context  for  discussion  of  risk 
items. 

Report  on  review 
actions 

Check  to  assure  ongoing  rigor  in  accomplishing  critical  risk 
activities. 

Review  status  of 
shared  government/ 
contractor  risks 

See  Section  4,  Managing  Risk  with  Others,  p.  55. 

New  risks:  identify, 
discuss,  prioritize,  and 
assign  actions. 

Everyone  is  focused  on  the  same  tomorrow.  Everyone  has 
the  same  definition  of  “success.”  Everyone  sees  the  final 
product  as  the  same  thing. 

2.  Loveland  Link,  Jo  Lee,  et  al.  Rollout  and  Installation  of  Risk  Management  at  the 
MINT  Directorate,  National  Reconnaissance  Office  (CMU/SEI-99-TR-009).  Pitts¬ 
burgh,  Pa.:  Software  Engineering  Institute,  Carnegie  Mellon  University,  1999. 
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Tools 


There  are  many  ways  to  present  risk  information  during  status  reviews.  The  following 
list  is  a  summary  of  the  methods  and  tools  presented  in  Appendix  C. 


Tool 

Description 

Mitigation  status 
report 

This  method  requires  compiling  data  using  textual 
information  and  graphics  (e.g.,  time  graphs,  bar  graphs)  to 
document  detailed  information  about  specific  risk 
mitigation  plans.  Mitigation  status  reports  are  used  to 
support  decisions.  The  format  of  the  report  and  the 
information  included  in  the  report  should  be  tailored  to  the 
needs  of  an  organization  [Clark  95]. 

Risk  information  sheet 

This  form  documents  information  about  a  risk,  and  is  similar 
to  a  software  trouble  or  problem  report.  It  is  used  to 
document  detailed  information  on  specific  risks  and  to 
support  decisions  [see  “Selected  Risk  Management  Forms” 
on  page  93]. 

Spreadsheet  risk 
tracking 

This  method  uses  spreadsheets  to  summarize  the  current 
statuses  of  all  risks  and  provides  a  way  to  monitor  project 
risks.  The  basic  process  involves  a  periodic  (e.g.,  weekly  or 
monthly)  update  and  review  of  the  risks.  Spreadsheet  risk 
tracking  reports  are  normally  included  as  read-ahead 
material  for  project  meetings,  where  the  reports  are 
reviewed  and  updated  as  appropriate  [see  “Selected  Risk 
Management  Forms”  on  page  93]. 

Stoplight  chart 

This  is  a  tool  that  is  used  to  summarize  the  statuses  of 
important  risks  and  their  mitigation  efforts.  The  charts  are 
effective  tools  for  reporting  risk  information  to  senior 
management.  Each  mitigation  plan  is  assigned  one  of  three 
conditions: 

•  green — indicates  that  the  plan  is  working  as  intended  and 
that  no  management  action  is  required 

•  yellow — indicates  that  the  plan  is  not  working  as  intended, 
although  no  management  action  is  required 

•  red — ^indicates  that  the  plan  is  not  working  and  that 
management  action  is  required  [see  “Selected  Risk 
Management  Forms”  on  page  93] 

Bar  graph 

This  type  of  graph  depicts  data  across  distinct  categories. 

Bar  graphs  highlight  changes  in  the  number  of  risks  in 
individual  categories  and  can  be  used  to  identify  trends 
[Brassard  89,  Hays  88,  Moran  90]. 

Time  correlation  chart 

This  type  of  graph  shows  the  relationship  of  one  metric  with 
respect  to  another  over  time.  Time  correlation  charts  are 
useful  for  identifying  the  trend  over  time  in  the  relationship 
of  two  metrics  [Brassard  89,  Hays  88,  Moran  90]. 

Time  graph 

This  type  of  graph  illustrates  data  variations  over  time. 

Time  graphs  are  useful  for  identifying  the  trend  over  time  of 
a  risk  metric  [Brassard  89,  Hays  88,  Moran  90]. 

Recommenda' 
tions  for 
Activity  7 
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•  Project  teams  should  sharpen  their  meeting  management  skills  to  improve  the 
productivity  of  reviews. 

•  Project  managers  and  team  members  should  develop  ground  rules  for  effective 
reviews. 

•  Project  managers  and  team  members  should  develop  and  enforce  acceptable  behavior 
guidelines  for  review  participants. 

•  Integrate  risk  functions  (identify,  analyze,  plan,  track,  control)  into  the  review  agenda. 

•  The  project  team  should  select  methods  and  tools  that  communicate  the  status  of 
known  risks. 
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Institutionalization  Features 


Institutionalization  features  are  the  building  blocks  for  corporate  culture  and  infrastruc¬ 
ture.  They  are  critical  to  successfully  implementing  the  Acquisition  Risk  Management 
KPA.  They  help  define  common,  corporate-wide  methods,  practices,  and  procedures 
that  become  the  ongoing  way  of  doing  business.  These  are  defined  in  such  a  way  that 
they  continue  even  after  those  who  originally  defined  them  are  gone. 
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Commitment  to  Perform 


Description 


Objective 


Commitment  1 
Description 

Objective 

Policy 

Components 


Example 


Commitment  to  Perform  describes  the  actions  that  the  organization  must  take  to  estab¬ 
lish  the  process  and  ensure  that  it  can  endure.  Commitment  to  Perform  typically 
involves  establishing  organizational  policies  and  management  sponsorship. 

The  objective  of  the  Commitment  to  Perform  institutionalization  feature  is  to  visibly 
communicate  a  freely  assumed  pact  that  is  expected  to  be  kept  by  all  parties.  A  written 
policy  statement  emphasizes  the  connection  between  organizational  commitment  and 
the  activities  performed  by  a  project.  Management  commitment  and  sponsorship  is  dis¬ 
played  by  designating  responsibility  and  accountability  for  actions. 

The  acquisition  organization  has  a  written  policy  for  the  management  of  software 
acquisition  risk. 

Commitment  1  requires  a  written  policy  at  the  acquisition  organization  level  describing 
how  projects  will  perform  software  acquisition  risk  management.  This  policy  will  apply 
to  all  projects  and  sets  the  stage  for  a  standardized  approach  to  risk  management. 

The  objective  of  Commitment  1  is  to  communicate  the  acquisition  organization’s  policy 
for  software  acquisition  risk  management  clearly  and  unambiguously  to  all  current  and 
future  members  of  the  organization. 

The  policy  should  describe  how  projects  are  to  identify  and  manage  risks  throughout  the 
project  life  cycle  in  accordance  with  the  project’s  defined  software  acquisition  process 
and  should  reflect  how  business  is  actually  conducted.  It  should  include  a  discussion  of 
how  the  project  team,  the  end  user,  and  the  contractor  interact  to  perform  risk  manage¬ 
ment  activities.  The  policy  should  state  that  risk  management  is  a  proactive  and  positive 
part  of  software  acquisition  and  should  describe  how  risk  information  is  communicated 
throughout  the  project  team. 

The  Department  of  Defense  (DOD)  recently  revised  and  streamlined  its  acquisition  reg¬ 
ulations  clarifying  mandatory  policy  and  decentralizing  acquisition  practice  [Myers 
96].  The  resulting  documentation  is  leaner  and  specifies  only  key  activities  that  the 
project  manager  of  a  major  defense  or  major  information  system  acquisition  must  per¬ 
form.  The  resulting  written,  highly  visible  policy  for  acquisition  risk  management  fol¬ 
lows: 


**The  PM  shall  establish  a  risk  management  program  for  each  acquisition  pro¬ 
gram  to  identify  and  control  performance,  cost,  and  schedule  risks.  The  risk 
management  program  shall  identify  and  track  risk  drivers,  define  risk  abate¬ 
ment  plans,  and  provide  for  continuous  risk  assessment  throughout  each  acqui¬ 
sition  phase  to  determine  how  risks  have  changed.  Risk  reduction  measures 
shall  be  included  in  cost-performance  trade-offs,  where  applicable.  The  risk 
management  program  shall  plan  for  back-ups  in  risk  areas  and  identify  design 
requirements  where  performance  increase  is  small  relative  to  cost,  schedule, 
arid  performance  risk.  The  acquisition  strategy  shall  include  identification  of 
the  risk  areas  of  the  program  and  a  discussion  of  how  the  PM  intends  to  man¬ 
age  those  risks”  [DOD  96]. 

An  acquisition  organization  needs  a  similar,  highly  visible  statement  detailing  its  policy 

for  software  acquisition  risk  management. 
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Managing  Risk 
with  Others 


Recommenda¬ 
tions  for  Com¬ 
mitment  1 


Commitment  2 
Description 


Objective 


Designating 

Responsibility 


Acquisition  risks  extend  beyond  the  project  team.  The  acquisition  organization  may 
have  multiple  projects  which  interact  to  mitigate  risks  of  a  global  nature.  A  project 
team  may  share  risk  with  its  contractor(s)  to  increase  the  effectiveness  of  risk  handling 
strategies.  Users  tend  to  be  a  major  source  of  risks  with  evolving  requirements  and  have 
an  important  voice  in  mitigation  options.  All  of  these  external  groups  are  vital  to  suc¬ 
cessful  software  acquisition  risk  management  and  the  policy  statement  from  the  acquisi¬ 
tion  organization  should  validate  their  role  in  the  identification,  analysis,  planning, 
tracking,  control,  and  communication  of  risks  (see  Section  4,  Managing  Risk  with  Oth¬ 
ers,  p.  55). 

•  The  acquisition  organization  should  have  a  written  policy  on  software  acquisition  risk 
management. 

•  The  policy  should  include 

•  a  discussion  of  the  importance  of  identifying  risks  throughout  the  acquisition 

•  a  discussion  of  inter-organizational  risk  management  activities,  including  how  the 
project  team,  contractor(s),  and  end  user(s)  are  involved  in  risk  management 
activities 

•  how  risk  information  is  communicated 

•  designation  of  responsibility  at  the  acquisition  organization  level  (see  Commitment 
2,p.46) 

•  a  clear  statement  validating  acquisition  risk  management  as  a  positive  and  proactive 
part  of  software  acquisition 

Responsibility  for  software  acquisition  risk  management  activities  is  designated. 

Commitment  2  clarifies  the  risk  management  functions  by  designating  responsibility  to 
acquisition  organization  and  project  team  members. 

The  objective  of  Commitment  2  is  to  increase  the  visibility  of  the  software  acquisition 
risk  management  activities  by  formally  designating  responsibility  for  risk  management 
functions.  Each  member  of  Ae  organization  should  understand  Aeir  responsibilities 
clearly  and  how  their  risk  management  activities  support  the  entire  process. 

Every  member  of  the  acquisition  organization  needs  to  understand  their  responsibilities 
for  performing  software  acquisition  risk  management.  There  are  many  ways  an  acquisi¬ 
tion  organization  can  designate  responsibilities  and  the  approach  should  be  based  on  the 
needs  of  the  acquisition  organization.  Fundamental  elements  are  that  it  should  be  in 
writing,  all  members  of  the  organization  should  understand  their  responsibilities,  and  it 
needs  to  reflect  how  responsibility  and  accountability  are  actually  designated.  Roles 
and  responsibilities  are  documented  in  the  Software  Acquisition  Risk  Management  Plan 
(see  Activity  2,  p.  28). 
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The  following  diagram  depicts  one  way  an  organization  can  designate  risk  management 
responsibility  [Dorofee  96]: 


In  the  above  example,  every  individual  in  the  organization  is  responsible  for  continu¬ 
ously  identifying,  evaluating  and  classifying  risks,  and  recommending  mitigation  plans. 
An  individud  or  group  may  be  assigned  the  responsibility  of  tracking  the  status  of  risks 
for  the  project  or  organization. 

In  the  above  example,  in  addition  to  their  responsibilities  as  individuals  and  team  mem¬ 
bers,  technical  leads  evaluate  and  classify  risks  and  approve  risk  mitigation  plans. 

In  the  above  example,  the  project  manager  reviews  and  integrates  risk  information, 
assigns  responsibility  for  risks  and  mitigation  plans,  and  handles  communications  exter¬ 
nal  to  the  project. 

A  small  software  acquisition  project  office  may  need  to  designate  responsibility  differ¬ 
ently  than  the  above  example.  The  project  manager  may  have  only  one  level  of  staff 
members  who  double  as  technical  leads  and  team  members.  Regardless  of  how  the 
project  designates  responsibility  for  software  acquisition  risk  management,  it  needs  to 
be  in  writing. 

•  Responsibility  for  software  acquisition  risk  management  activities  should  be  formally 
designated  at  both  the  acquisition  organization  and  project  levels. 

•  Projects  should  document  roles  and  responsibilities  in  the  Software  Acquisition  Risk 
Management  Plan  (see  Activity  2,  p.  28). 

•  At  the  acquisition  organization  level,  responsibility  should  be  designated  in  the  policy 
statement  (see  Commitment  1,  p.  45). 
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Ability  to  Perform 


Ability  to  Perform  describes  the  preconditions  that  must  exist  in  the  project  or  organiza¬ 
tion  to  implement  the  software  acquisition  process  competently.  Ability  to  Perform  typ¬ 
ically  involves  resources,  organizational  structures,  and  training. 

The  objective  of  the  Ability  to  Perform  institutionalization  feature  is  to  ensure  that  the 
organization  has  all  of  the  resources,  organizational  structures,  and  training  required  to 
acquire  software.  Resources  include  access  to  special  skills,  adequate  funding,  and 
tools.  Organizational  structures  are  described  to  ensure  that  capability  exists  to  perform 
the  KPA.  Training  can  include  both  formal  and  informal  methods  of  transferring  knowl¬ 
edge  to  individuals  in  the  organization.  Training  requirements  are  identified  and  pro¬ 
vided  through  the  Training  Program  KPA. 

A  group  that  is  responsible  for  coordinating  software  acquisition  risk  management 
activities  exists. 

Ability  1  ensures  that  adequate  personnel  are  available  to  coordinate  software  acquisi¬ 
tion  risk  management  functions. 

The  objective  of  Ability  1  is  to  ensure  that  a  collection  of  departments,  managers,  and 
individuals  exists  to  perform  the  acquisition  risk  management  tasks  and  activities. 
Clearly  identifying  a  group  responsible  for  coordinating  these  activities  shows  organiza¬ 
tional  commitment  beyond  a  policy  statement  and  communicates  the  importance  of  per¬ 
forming  acquisition  risk  management. 

In  the  S A-CMM,  a  group  may  vary  from  a  single  individual  assigned  part-time,  to  sev¬ 
eral  part-time  individuals  matrixed  from  other  organizations,  to  severd  individuals  ded¬ 
icated  full-time.  The  acquisition  organization  should  define  “group”  based  on  the  needs 
of  the  individual  projects.  Candidate  members  of  the  group  could  come  from  the  project 
team,  the  user  community,  and  the  contractor(s). 

There  are  three  basic  approaches  to  ensure  that  a  group  to  coordinate  risk  management 
activities  exists.  In  a  survey  of  DOD  program  management  offices,  one  set  of  respon¬ 
dents  allocated  specific  positions  to  coordinate  risk  management  activities  while  a  sec¬ 
ond  set  felt  that  risk  management  was  so  integral  to  project  management  that  separate 
personnel  were  not  identified  [DSMC  89].  A  third  approach  is  to  identify  specific  per¬ 
sonnel  for  some  of  the  administrative  functions  while  recognizing  that  all  project  per¬ 
sonnel  must  participate  in  risk  management  (see  Commitment  2,  p.  46).  Whichever 
approach  is  selected,  the  acquisition  organization  needs  to  ensure  that  personnel 
accountability  decisions  are  documented. 

•  The  acquisition  organization  should  ensure  that  adequate  personnel  are  available  to 
each  project  to  perform  the  acquisition  risk  management  activities. 


Adequate  resources  are  provided  for  software  acquisition  risk  management  activi¬ 
ties. 

Ability  2  requires  the  acquisition  organization  to  provide  adequate  resources  to  perform 
the  software  acquisition  risk  management  functions.  The  required  resources  are  docu¬ 
mented  in  the  Software  Acquisition  Risk  Management  Plan  (see  Activity  2,  p.  28). 
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The  objective  of  Ability  2  is  to  ensure  that  the  funding,  staff,  equipment,  and  tools 
required  to  perform  acquisition  risk  management  are  available  to  the  project.  This 
allows  individual  projects  to  successfully  implement  and  maintain  a  healthy  acquisition 
risk  management  program  that  continues  throughout  the  life  of  the  project. 

The  following  table  describes  risk  management  resources: 


Resource 

Description 

Example 

Funding 

Specific  monies  may  be 
required  to  adequately  perform 
acquisition  risk  management  on 
the  project.  Funding  should  be 
set  aside  for  resources  that  the 
project  doesn’t  already  have. 

The  project  may  want  to  use  a 
relational  database  tool  to  store 
risk  information.  Funding  should 
be  allocated  to  procure  and 
support  the  database  tool  and 
associated  computer  systems. 

Staff 

Project  team  members  can 
perform  all  risk  management 
functions  as  an  integral  part  of 
their  project  duties  (see 
Conunitment  2,  p.  46).  If 
additional  personnel  are 
required  to  effectively  perform 
the  project’s  defined 
acquisition  risk  management 
process,  people  should  be 
provided  to  the  project. 

A  project  may  need  a  technical 
assistant  to  help  coordinate  the 
acquisition  risk  management 
activities  (see  Ability  1,  p.  48). 

Equipment 

Equipment  encompasses 
everything  from  pencil  and 
paper,  dry-erase  boards, 
overheads,  and  adequate 
facilities,  to  computer  systems 
to  store  and  track  risk  data. 

Project  team  members  need  to 
capture  risks  they  identify  (see 
Activity  1,  p.  25).  The 
acquisition  organization  needs  to 
supply  personnel  with  equipment 
which  facilitates  and  encourages 
risk  identification. 

Tools 

Tools  include  a  defined  process 
and  any  system,  either  manual 
or  automated,  allowing  project 
members  to  perform  the  risk 
management  functions  (see 
Appendix  C,  p.  79). 

The  project  may  use  a  time  chart 
or  run  graph  to  document  the 
values  of  risk  status  metrics  over 
time. 

One  project  actively  solicited  risk  data  from  project  members  using  a  standard  risk  iden¬ 
tification  form.  Due  to  inadequate  resources,  the  project  manager  had  no  easy  way  to 
track  the  risks  and  placed  them  in  her  filing  cabinet.  During  operational  acceptance  of 
the  system,  a  real-time  processor  couldn’t  handle  the  amount  of  data  encountered  in  the 
operational  environment,  requiring  a  major  re-write  of  the  software  and  an  upgrade  of 
the  processor.  This  problem  was  encountered  during  operational  testing  but  was  identi¬ 
fied  as  a  risk  early  in  system-level  design.  The  re-work  could  have  been  avoided  if  the 
project  manager  had  been  given  the  resources  to  track  and  mitigate  risks. 
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•  The  acquisition  organization  should  ensure  that  projects  have  adequate  funding,  staff, 
equipment,  and  tools  to  perform  acquisition  risk  management  activities. 

•  Resources  required  to  perform  the  software  acquisition  risk  management  functions 
should  be  documented  in  the  Software  Acquisition  Risk  Management  Plan  and  should 
be  updated  as  necessary  throughout  the  project  to  reflect  changing  needs. 

Individuals  performing  software  acquisition  risk  management  activities  have  expe¬ 
rience  or  receive  required  training. 

Ability  3  requires  that  the  acquisition  organization  and  each  project  have  individuals 
who  know  how  to  perform  software  acquisition  risk  management.  At  Level  3  of  the  SA- 
CMM,  training  needs  of  the  acquisition  organization  and  project  are  identified  and  satis¬ 
fied  by  the  Training  Program  KPA. 

The  objective  of  Ability  3  is  to  ensure  that  each  project  has  team  members  performing 
software  acquisition  risk  management  who  are  competent  in  software  acquisition,  risk 
management,  and  the  problem  domain. 

Team  members  assigned  to  perform  acquisition  risk  management  functions  on  a  project 
need  to  understand  the  principles  of  software  acquisition,  how  to  perform  the  acquisition 
organization’s  risk  management  process,  as  well  as  the  domain  or  problem  space  in 
which  the  application  solution  needs  to  operate.  Individuals  need  to  have  knowledge  in 
all  three  areas  to  successfully  contribute  to  the  project. 

The  following  table  describes  methods  of  obtaining  required  knowledge: 


Method 

Description 

Experience 

An  individual  has  the  experience  required  to  perform  software 
acquisition  risk  management  if  the  following  is  true:  the  individual 
has  participated  in  a  software  acquisition  management  role  on  at  least 
one  project,  has  applied  risk  management  techniques  on  at  least  one 
project,  and  has  experience  in  the  domain  of  the  application  being 
acquired  [Fisher  99].  If  the  individual  is  lacking  in  any  of  the  above, 
he  or  she  should  be  afforded  training  in  that  area. 

Formal 

Training 

Formal  classroom  training  is  provided  by  the  acquisition 
organization  or  a  third  party.  This  allows  the  project  members  to  feel 
confident  in  performing  their  risk  management  activities. 

Informal 

Training 

Informal  training  may  include  self-paced  courses  or  on-the-job 
training  (OJT)  through  the  use  of  a  strong  mentoring  program. 

Note:  OJT  does  not  mean  that  an  individual  is  assigned 
responsibility  and  expected  to  perform.  A  successful  OJT  program 
includes  mentors  who  are  responsible  for  results  while  trainees  learn. 

•  The  acquisition  organization  should  provide  knowledgeable  personnel  to  project  teams 
to  perform  acquisition  risk  management  activities. 

•  A  written  plan  (e.g.,  the  project’s  Training  Plan)  should  specify  the  risk  management 
training  required  for  project  personnel  and  should  specify  the  training  schedule. 
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Measurement  and  Analysis 


Measurement  and  Analysis  describes  the  need  to  measure  the  process  and  analyze  the 
measurements.  Measurement  and  analysis  typically  includes  examples  of  the  measure¬ 
ments  that  could  be  taken  to  determine  the  status  and  effectiveness  of  the  activities  per¬ 
formed. 

The  objective  of  Measurement  and  Analysis  is  to  collect  data  which  is  then  used  to  con¬ 
trol  and  improve  the  way  the  acquisition  organization  and  its  projects  conduct  business. 
The  measurements  taken  should  be  based  on  the  goals  of  the  projects  [Park  96].  Vari¬ 
ability  in  project  environments  may  lead  to  different  measurement  needs  and 
approaches.  There  are  currently  no  universally  accepted  measures  of  software  acquisi¬ 
tion  process  or  quality  [Paulk  95]. 

Measurements  are  made  and  used  to  determine  the  status  of  the  acquisition  risk 
management  activities  and  resultant  products. 

Measurement  1  requires  the  project  to  measure  the  software  acquisition  risk  manage¬ 
ment  process  and  die  quality  of  the  products  produced  by  the  process. 

The  objective  of  Measurement  1  is  to  focus  management  attention  on  the  process  that 
the  acquisition  organization  and  individual  projects  use  to  conduct  software  acquisition 
risk  management.  A  process  focus  allows  managers  to  identify  weaknesses  in  the  way 
they  do  business  and  helps  them  improve  the  quality  of  their  products  by  improving  the 
process  that  produces  them. 

When  a  risk  is  identified  initially,  project  team  members  will  usually  provide  an  initial 
indication  of  the  probability  of  its  occurrence  and  the  impact  to  the  project  if  it  happens 
(see  Section  1.1,  p.  3).  This  combination  of  probability  and  impact  provides  a  level  of 
exposure  for  a  given  risk.  As  the  project  team  takes  specific  actions  to  mitigate  the  risk, 
or  as  project  objectives  or  conditions  change,  the  risk  exposure  will  change  accordingly. 
Since  the  level  of  exposure  for  a  given  risk  changes  over  time,  the  project  team  should 
periodically  re-evaluate  the  risk  attributes  (probability,  impact)  to  determine  the  risk’s 
importance  to  the  project. 

Mitigation  plans  can  be  simple  action  item  lists  or  elaborate  plans.  As  with  any  acquisi¬ 
tion  planning,  the  project  team  should  evaluate  how  well  they  are  performing  to  the  plan 
and  revise  the  plan  as  necessary.  Note  that  a  team  may  execute  a  plan  flawlessly,  but  the 
risk  exposure  for  a  given  risk  may  not  change.  Because  of  this,  it  is  important  to  mea¬ 
sure  risk  and  progress  toward  mitigation  plans  separately. 
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Metrics  to  monitor  the  risk  management  process  are  defined  early  in  the  project’s  life 
cycle  and  are  documented  in  the  Software  Acquisition  Risk  Management  Plan  (see 
Activity  2,  p.  28).  A  project  team  may  use  a  top  N  list  to  track  the  highest-priority  risks 
associated  with  a  given  phase  of  the  project.  Collecting  and  using  risk  management  pro¬ 
cess  metrics  allows  the  project  team  to  identify  weaknesses  in  how  the  top  N  list  is 
developed  and  maintained  and  provides  the  opportunity  to  improve  the  way  they  handle 
this  critical  data.  Other  examples  of  measurements  include  [Dorofee  96] 

•  number  of  risks  open 

•  number  of  risks  by  classification 

•  trends  in  risk  processing  from  identification  to  closure 

•  number  of  successful  mitigations  versus  failed  mitigations 

Note:  See  Activity  6,  p.  37,  for  a  discussion  on  how  the  risks  themselves  are  tracked  and 
measured. 

Example  One  project  manager  decided  to  measure  which  organizations  were  actively  submitting 

risks  for  his  project,  because  anecdotal  data  told  him  that  important  operational  voices 
weren’t  being  heard.  The  results  showed  that  60%  of  the  risks  were  identified  by  the 
acquisition  project  team,  38%  were  identified  by  the  contractor,  and  only  2%  were  iden¬ 
tified  by  the  operational  user.  This  hard  data  helped  the  project  manager  focus  resources 
on  finding  a  root  cause.  After  a  review  of  the  risk  identification  process,  the  project 
manager  found  that  the  operational  users’  organization  had  imposed  a  lengthy  and  sti¬ 
fling  process  that  users  had  to  navigate  in  order  to  voice  potential  problems.  The  users 
were  so  disgusted  with  the  process  that  they  quit  formally  identifying  risks  and  waited  to 
bring  up  their  “issues”  during  system  testing.  The  users’  organization  was  quickly 
trained  on  the  project’s  defined  software  acquisition  risk  management  process  and 
encouraged  to  remove  the  barriers. 

•  The  project  team  should  measure  the  status  of  individual  risks,  their  mitigation  plans, 
and  the  risk  management  process  to  determine  the  status  of  the  acquisition  risk 
management  activities. 

•  The  project  team  should  use  the  results  of  measurements  as  a  basis  for  project 
management  and  acquisition  organization  management  verifications  (see  Verification 
1,  p.  53,  and  Verification  2,  p.  54). 
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Verifying  Implementation  describes  the  steps  to  ensure  that  the  activities  are  performed 
in  compliance  with  the  process  that  has  been  established.  Verifying  Implementation 
typically  encompasses  reviews  by  management. 

The  objective  of  Verifying  Implementation  is  to  provide  senior  management  and  project 
management  insight  into  the  activities  performed  by  the  project  team  to  ensure  compli¬ 
ance  with  the  acquisition  organization’s  standard  software  acquisition  risk  management 
process  and  the  project’s  defined  process.  Review  of  these  activities  by  management 
indicates  the  importance  that  the  organization  places  on  the  acquisition  risk  management 
process. 

Acquisition  risk  management  activities  are  reviewed  by  acquisition  organization 
management  on  a  periodic  basis. 

Verification  1  requires  acquisition  organization  management  (senior  management  above 
the  project  manager)  to  review  acquisition  risk  management  activities  on  a  periodic 
basis. 

The  objective  of  Verification  1  is  to  provide  awareness  of  and  insight  into  the  software 
acquisition  risk  management  process  activities  to  senior  managers.  The  information 
should  be  communicated  at  an  appropriate  level  of  abstraction  and  in  a  timely  manner. 

The  time  between  reviews  for  acquisition  organization  management  may  be  lengthy  as 
long  as  adequate  mechanisms  are  established  for  exception  reporting.  The  scope  and 
content  may  vary  depending  on  what  the  acquisition  organization  management  wants  to 
see.  Anticipate  that  acquisition  organization  management  will  expect  different  data  than 
the  project  manager  or  the  same  data  at  a  higher  level  of  abstraction  (e.g.,  the  project’s 
top  risks,  rate  of  risk  identification  versus  mitigation). 


The  following  diagram  shows  the  acquisition  risk  management  process  as  seen  at  the 
acquisition  organization  level. 
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•  The  project  team  should  present  the  results  of  the  acquisition  risk  management 
activities  to  acquisition  organization  management  at  periodic  program  reviews. 

•  The  project  team  should  present  the  status  of  the  project’s  top  risks  and  mitigation 
plans. 

•  The  project  team  should  present  data  that  indicate  the  effectiveness  of  the  acquisition 
risk  management  process  (e.g.,  rate  of  identification  versus  rate  of  mitigation). 

Acquisition  risk  management  activities  are  reviewed  by  the  project  manager  on 
both  a  periodic  and  event-driven  basis. 

Verification  2  requires  the  project  manager  to  review  software  acquisition  risk  manage¬ 
ment  activities  both  on  a  periodic  and  event-driven  basis. 

The  objective  of  Verification  2  is  to  ensure  that  project  managers  maintain  an  ongoing 
awareness  of  the  status  of  the  software  acquisition  risk  management  efforts  and  receive 
information  when  significant  events  occur. 

Oversight  at  the  project  manager  level  varies  depending  on  the  characteristics  of  the 
project  and  the  needs  of  the  project  manager.  The  key  is  awareness  which  comes  from 
participation  in  formal  reviews,  such  as  periodic  project  management  reviews  and  staff 
meetings,  as  well  as  informal  reviews  such  as  real-time  status  reports  and  reviews  of 
non-compliance  issues.  The  reviews  at  the  project  manager  level  should  be  more 
detailed  Aan  those  of  the  acquisition  organization  management  because  the  project 
manager  takes  a  more  active  role  in  the  operational  aspects  of  the  project  [Paulk  95]. 

A  project  manager  of  a  large,  software-intensive  project  has  integrated  risk  management 
into  his  project  management  activities  and  has  replaced  periodic  program  reviews  with 
reviews  of  risk  data.  The  project  no  longer  spends  days  pouring  over  every  aspect  of  the 
project.  Rather,  the  results  of  the  project’s  risk  management  process  are  reviewed  in  a 
proactive  approach  to  project  management. 

•  The  project  manager  should  review  and  participate  in  the  acquisition  risk  management 
activities. 
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A  fundamental  principle  of  acquisition  risk  management  is  that  you  can  successfully 
manage  risk  only  if  you  are  proactive  and  forward  thinking.  The  practices  and  goals  of 
the  ARM  KPA  provide  mechanisms  and  techniques  that  foster  this  proactive  principle. 
Software  acquisition  includes  acquirer,  developer,  and  user  organizations,  among  others. 
In  a  traditional  model,  acquisition  includes  two  organizations  (i.e.,  an  acquiring  and  a 
developing  organization)  or  two  divisions  within  the  same  organization  informally  con¬ 
tracting  for  products  and  services.  The  users  of  the  acquired  software  could  be  part  of  a 
third  organization.  The  relationship  between  the  entities  is  the  important  aspect  with 
regard  to  software  acquisition  risk  management.  In  the  simplest  scenario,  the  develop¬ 
ment  organization  is  usually  “contracted”  to  do  the  development  within  certain  cost, 
schedule,  and  performance  parameters  dictated  by  the  contract.  The  acquisition  organi¬ 
zation  traditionally  manages  the  contract  and  ensures  that  the  requirements  for  the 
development  are  being  met.  The  users  provide  the  requirements  for  the  performance 
and  functionality  of  the  system.  Risk  management  must  be  performed  by  all  organiza¬ 
tions  on  a  continuous  basis;  it  can  be  performed  jointly  when  risks  affect  more  than  one 
party.  When  diverse  organizations  form  a  team  to  achieve  common  goals,  the  team’s 
effectiveness  can  be  greater  than  the  sum  of  each  organization’s  individual  effective¬ 
ness. 

Managing  risk  with  other  organizations  depends  on  systematic  and  continuous  risk  man¬ 
agement  processes  in  the  individual  organizations.  All  organizations  must  work 
together  to  cooperatively  manage  risk  throughout  the  project’s  life  cycle  [Gluch  95]. 
The  result  is  a  disciplined  environment  for  proactive  decision-making  among  two  or 
more  organizations.  This  is  accomplished  using  a  structure  where  personnel  from  multi¬ 
ple  organizations  work  together  to  share  information  about  risks  that  may  affect  the 
other  organization(s).  Creating  such  a  structure  requires  a  common  work  culture,  a 
common  set  of  motivators,  and  an  emphasis  on  communication  among  the  organiza¬ 
tions. 

Teaming  with  external  organizations  to  perform  risk  management  activities  might 
require  a  shift  in  paradigms.  The  project  team  and  external  groups  must  work  together 
as  a  team  and  it  might  take  time  to  build  trust  among  personnel  from  different  organiza¬ 
tions.  A  strong  and  visible  statement  from  a  project’s  sponsor  is  often  required  to  vali¬ 
date  this  approach  and  to  encourage  the  formation  of  teams  at  the  inter-organizational 
level  (see  Commitment  1,  p.  45). 

Continuous  identification  of  risks  is  generally  left  to  the  individual  organizations 
through  the  use  of  their  defined  risk  management  process.  Risks  that  are  identified  by 
an  organization  might  be  appropriately  re-worded  for  an  inter-organizational  audience. 
It  is  also  possible  for  inter-organizational  teams  to  identify  risks  using  techniques  tai¬ 
lored  from  the  parent  organizations’  standard  practices. 

Evaluation  and  classification  of  risks  primarily  depends  on  each  organization’s  risk 
management  process.  The  main  task  at  the  inter-organizational  level  is  to  prioritize 
risks,  resulting  in  a  joint  list  of  risks  that  are  most  important  to  the  program.  A  joint  list 
of  risks  identifies  the  risks  for  which  mitigation  planning  must  be  performed  with  input 
from  multiple  organizations. 
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Summary 


To  the  extent  possible,  planning  for  all  risks  should  be  performed  within  an  organiza¬ 
tion.  In  special  cases,  planning  is  performed  at  the  inter-organizational  level  through 
joint  action  planning.  For  joint  risks,  the  inter-organizational  team  should  delegate  plan¬ 
ning  to  one  of  the  organizations  if  possible. 

Joint  action  planning  involves  sharing  data  across  organizations  as  well  as  using  exper¬ 
tise  and  knowledge  from  all  organizations  effectively.  Risks  that  are  identified  by  an 
inter-organizational  team  require  effort  from  all  involved  organizations  for  effective  mit¬ 
igation. 

Note:  Because  joint  action  planning  involves  all  parties,  outside  facilitation  may  be 
required. 

The  organization  with  primary  responsibility  for  a  risk  acquires  and  compiles  tracking 
data  and  reports  risk  and  mitigation  plan  status.  Control  decisions  are  made  by  person¬ 
nel  within  the  organization  responsible  for  the  risk. 


Product  development  generally  requires  the  specialization  and  division  of  activities 
required  to  make  a  product.  Integrated  product  development  (IPD)  is  a  model  for  prod¬ 
uct  development  in  which  activities  requiring  multiple  disciplines  are  integrated  and 
adapted  for  the  development  task.  Rapidly  changing  technology,  short  development 
cycles,  and  the  need  for  various  types  of  expertise  required  to  develop  a  single  product 
create  a  need  for  IPD  [Andreasen  85].  As  a  result,  IPD  is  common  in  today’s  product 
development  environments.  Applied  to  the  acquisition  process,  integrated  product 
development  teams  (IPDTs)  comprise  team  members  possessing  skills  from  multiple 
disciplines  who  collaborate  throughout  the  acquisition  life  cycle  to  develop  a  system  or 
component  of  a  system.  Depending  on  circumstances,  team  members  might  represent  a 
single  organization  or  company,  or  they  might  be  from  different  organizations  or  compa¬ 
nies.  In  either  case,  an  IPDT  uses  systematic  and  continuous  risk  management  pro¬ 
cesses  as  part  of  the  team’s  project  activities.  If  an  IPDT  must  interface  with  other 
IPDTs  as  part  of  a  larger  development,  then  all  of  the  IPDTs  must  work  together  to  coop¬ 
eratively  manage  risk  throughout  the  project’s  life  cycle.  The  result  is  a  disciplined 
environment  for  risk  management  among  two  or  more  IPDTs. 


An  organization  is  acquiring  engineering  and  manufacturing  information  systems  as  part 
of  an  initiative  to  update  its  design  and  production  processes.  An  IPDT  was  chartered 
to  develop  the  interface  between  the  engineering  and  manufacturing  systems.  This 
cross-functional  IPDT  comprised  members  of  the  acquisition  project  team  for  each  sys¬ 
tem  and  members  of  the  system  engineering  and  development  organizations  from  each 
contractor  organization.  As  part  of  the  interface  development,  the  IPDT  continuously 
manages  risk.  Any  risks  that  affect  either  the  manufacturing  or  production  system 
development  or  any  other  related  system  development  are  managed  cooperatively  with 
the  IPDTs  developing  those  systems. 

Joint  events  to  manage  and  discuss  risks  will  bring  together  personnel  from  different 
organizations.  The  benefits  of  inter-organizational  risk  management  are 

•  elevation  of  risks  facing  a  project  to  a  level  which  allows  them  to  be  handled  by 
personnel  from  all  of  the  involved  organizations 

•  creation  of  a  framework  to  implement  integrated  management  and  continuous  process 
principles 

•  creation  of  long-term  inter-organizational  relationships  based  on  trust 
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Glossary 


accept-A  mitigation  approach  that  essentially  does  nothing  with  the  risk.  It  is  handled 
as  a  problem  if  it  occurs.  No  risk  management  resources  are  expended  dealing  with 
accepted  risks.  (See  acceptance  rationale.) 

acceptance  rationale-A  type  of  action  plan  that  documents  the  reason  or  rationale  for 
accepting  a  risk  (doing  noAing  with  it).  This  is  documented  for  historical  reasons. 

acquisition-The  process  of  obtaining  through  contract. 

acquisition  organization-That  entity  which  has  the  oversight  responsibility  for  Ae 
software  acquisition  project  and  which  may  have  purview  over  Ae  acquisition  activities 
of  a  number  of  projects  or  contract  actions. 

acquisition  organization’s  standard  software  acquisition  process-(See  software 
acquisition  process.) 

activity-Any  step  taken  or  function  performed,  eiAer  mental  or  physical,  toward 
achieving  some  objective.  Activities  include  all  Ae  work  Ae  managers  and  technical 
staff  do  to  perform  the  tasks  of  the  project  and  organization. 

Analyze-One  of  Ae  six  functions  of  Ae  SEI  risk  management  paradigm.  The  Analyze 
function  is  a  process  in  which  risks  are  examined  in  further  detail  to  determine  the  extent 
of  Ae  risks,  how  they  relate  to  each  oAer,  and  which  ones  are  Ae  most  important  to  deal 
wiA.  Analyzing  risl^  has  Aree  basic  activities: 

•  evaluating  Ae  attributes  of  risks 

•  classifying  risks 

•  prioritizing  (ranking)  risks 

application  domain~A  bounded  set  of  related  systems  (i.e.,  systems  that  address  a  par¬ 
ticular  type  of  problem).  Development  and  maintenance  in  an  application  domain  usu¬ 
ally  require  special  skills  and/or  resources.  Examples  include  payroll  and  personnel 
systems,  avionics,  command  and  control  systems,  compilers,  and  expert  systems. 

attributes  (of  software)--Characteristics  of  software  such  as  reliability,  maintainability, 
portability,  and  complexity.  These  characteristics  are  sometimes  referred  to  as  quality 
attributes. 

baseline-A  specification  or  product  Aat  has  been  formally  reviewed  and  agreed  upon, 
Aat  Aereafter  serves  as  Ae  basis  for  furAer  development,  and  Aat  can  be  changed  only 
Arough  formal  change  control  procedures. 

capability  maturity  model-A  description  of  Ae  stages  through  which  organizations 
evolve  as  Aey  define,  implement,  measure,  control,  and  improve  Aeir  processes.  The 
model  provides  a  guide  for  selecting  process  improvement  strategies  by  facilitating  Ae 
determination  of  current  process  capabilities  and  the  identification  of  the  issues  most 
critical  to  quality  and  process  improvement. 

commitment-A  pact  Aat  is  freely  assumed,  visible,  and  expected  to  be  kept  by  all  par¬ 
ties. 
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common  features-The  subdivision  categories  of  the  SA-CMM  key  process  areas.  The 
common  features  are  attributes  that  indicate  whether  the  implementation  and  institution- 
alization  of  a  key  process  area  can  be  effective,  repeatable,  and  lasting.  The  SA-CMM’s 
common  features  are  the  following:  Commitment  to  Perform,  Ability  to  Perform,  Activ¬ 
ities  Performed,  Measurement  and  Analysis,  and  Verifying  Implementation. 

Communicate-One  of  the  six  functions  of  the  SEI  risk  management  paradigm.  The 
Communicate  function  is  a  process  in  which  risk  information  is  conveyed  between  all 
levels  of  a  project  team.  Risk  communication  deals  with  the  ideas  of  probability  and 
negative  consequences.  It  is  present  in  all  of  the  other  functions  of  the  SEI  risk  manage¬ 
ment  paradigm  and  is  essential  for  the  management  of  risks  within  an  organization. 
Communication  must  both  fit  within  an  organization’s  culture  and  expose  the  risks  that 
are  present  in  an  organization’s  projects. 

condition-The  key  circumstances,  situations,  etc.,  that  are  causing  concern,  doubt,  anx¬ 
iety,  or  uncertainty.  In  a  risk  statement,  the  condition  phrase  is  the  phrase  at  the  begin¬ 
ning  of  the  statement. 

consequence-The  possible  negative  outcomes  of  the  current  conditions  that  are  creating 
uncertainty.  In  a  risk  statement,  the  consequence  phrase  is  the  phrase  at  the  end  of  the 
statement. 

consistency-The  degree  of  uniformity,  standardization,  and  freedom  from  contradiction 
among  the  documents  or  parts  of  a  system  or  component. 

context  -dontext  provides  additional  detail  regarding  the  events,  circumstances,  and 
interrelationships  within  the  project  that  may  affect  the  risk.  This  description  is  more 
detailed  than  can  be  captured  in  the  basic  statement  of  risk. 

Continuous  Risk  Management-Continuous  Risk  Management  is  an  engineering  prac¬ 
tice  with  processes,  methods,  and  tools  for  managing  risks  in  a  project.  It  provides  a  dis¬ 
ciplined  environment  for  proactive  decision-making  to 

•  Assess  continuously  what  could  go  wrong  (risks). 

•  Determine  which  risks  are  important  to  deal  with. 

•  Implement  strategies  to  deal  with  those  risks. 

contract-A  binding  agreement  between  two  or  more  parties  that  establishes  the  require¬ 
ments  for  the  products  and  services  to  be  acquired. 

contract  integrity-The  adherence  and  compliance  to  contractual  and  legal  policies, 
regulations,  and  other  guidance. 

contract  terms  and  conditions-The  stated  legal,  financial,  and  administrative  aspects 
of  a  contract. 

contractor-The  entity  delivering  the  product  or  performing  the  service  being  acquired, 
even  if  that  entity  is  part  of  the  acquiring  organization. 

Control-One  of  the  six  functions  of  the  SEI  risk  management  paradigm.  The  Control 
function  is  a  process  that  takes  the  tracking  status  reports  for  the  watched  and  mitigated 
project  risks  and  decides  what  to  do  with  them  based  on  the  reported  data.  The  person 
who  has  accountability  for  a  risk  normally  makes  the  control  decision  for  that  risk. 
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The  general  process  of  controlling  risks  includes 

•  analyzing  the  status  reports 

•  deciding  how  to  proceed 

•  executing  the  decisions 

defined  leveI-(See  maturity  level.) 

deHned  software  acquisition  process-(See  software  acquisition  process.) 
effective-Adequate  to  accomplish  the  intended  purpose. 

end  user-The  individual  or  group  who  will  use  the  system  for  its  intended  operational 
use  when  it  is  deployed  in  its  environment. 

end  user  representatives-A  selected  sample  of  end  users  who  represent  the  total  popu¬ 
lation  of  end  users. 

evaluation-The  use  of  reviews,  inspections,  and/or  tests,  to  determine  that  a  software 
product  or  service  satisfies  specified  requirements. 

event-driven  basis-A  review  that  is  performed  based  on  the  occurrence  of  an  event 
within  the  project  (e.g.,  a  formal  review  or  the  completion  of  a  life  cycle  stage).  (See 
periodic  review  for  contrast.) 

findings-The  conclusions  of  an  assessment,  evaluation,  audit,  or  review  that  identify 
the  most  important  issues,  problems,  or  opportunities  within  the  area  of  investigation. 

funcdon-A  set  of  related  actions,  undertaken  by  individuals  or  tools  that  are  specifi¬ 
cally  assigned  or  fitted  for  their  roles,  to  accomplish  a  set  purpose  or  end. 

goals-The  aggregate  result  achieved  by  the  effective  implementation  of  the  common 
features  of  a  key  process  area.  The  goals  signify  the  scope  and  intent  of  each  key  process 
area. 

group-An  assemblage  of  personnel  organized  to  serve  a  specific  purpose  or  accomplish 
a  task.  A  group  may  vary  from  a  single  individual  assigned  part  time,  to  several  part- 
time  individuals  assigned  from  other  organizations,  to  several  individuals  dedicated  full¬ 
time. 

Identify-One  of  the  six  functions  of  the  SEI  risk  management  paradigm.  The  Identify 
function  is  a  process  of  transforming  uncertainties  and  issues  about  the  project  into  dis¬ 
tinct  (tangible)  risks  that  can  be  described  and  measured.  Identifying  risks  involves  two 
activities: 

•  capturing  a  statement  of  risk 

•  capturing  the  context  of  a  risk 

impact-The  loss  or  effect  on  the  project  if  the  risk  occurs.  Impact  is  one  of  the  three 
attributes  of  a  risk. 

infrastructure  costs-Those  costs  associated  with  implementing  risk  management  activ¬ 
ities  and  supporting  risk  management  processes,  methods,  and  tools  within  the  organiza¬ 
tion.  These  costs  may  be  spread  out  across  multiple  projects.  See  also  mitigation  costs 
and  risk  management  costs. 


initial  level-(See  maturity  level.) 

institutionalization-The  building  of  infrastructure  and  corporate  culture  that  supports 
methods,  practices,  and  procedures  so  that  they  are  the  ongoing  way  of  doing  business, 
even  after  those  who  originally  defined  them  are  gone. 

key  process  area-A  cluster  of  related  activities  in  an  area  of  software  acquisition  that, 
when  performed  collectively,  achieve  a  set  of  goals  considered  important  for  establish¬ 
ing  process  capability  in  that  area.  The  key  process  areas  have  been  defined  to  reside  at 
a  single  maturity  level.  These  are  the  principal  building  blocks  to  help  determine  the 
software  acquisition  process  capability  of  an  organization  and  understand  the  improve¬ 
ments  needed  to  advance  to  higher  maturity  levels. 

life  cycle-(See  software  life  cycle.) 

managed  and  controlled-Implies  that  the  version  of  the  work  product  in  use  at  a  given 
time  (past  or  present)  is  known  (i.e.,  version  control),  and  changes  are  incorporated  in  a 
controlled  manner  (i.e.,  change  control). 

manager-A  role  that  encompasses  providing  technical  and  administrative  direction  and 
control  to  individuals  performing  tasks  or  activities  within  the  manager's  area  of  respon¬ 
sibility.  The  traditional  functions  of  a  manager  include  planning,  resourcing,  organiz¬ 
ing,  directing,  and  controlling  work  within  an  area  of  responsibility. 

maturity  level-A  well-defined  evolutionary  plateau  toward  achieving  a  mature  soft¬ 
ware  acquisition  process.  The  five  maturity  levels  in  the  SA-CMM  are  Initial,  Repeat- 
able,  Defined,  Quantitative,  and  Optimizing. 

measure-To  ascertain  the  characteristics  or  features  (extent,  dimension,  quantity,  capac¬ 
ity,  and  capability)  of  something,  especially  by  comparing  with  a  standard. 

measurement-The  dimension,  capacity,  quantity,  or  amount  of  something  (e.g.,  300 
source  lines  of  code  or  seven  document  pages  of  design). 

method-A  reasonably  complete  set  of  rules  and  criteria  that  establishes  a  precise  and 
repeatable  way  of  performing  a  task  and  arriving  at  a  desired  result. 

methodology-A  collection  of  methods,  procedures,  and  standards  that  defines  an  inte¬ 
grated  synthesis  of  approaches. 

metric-A  standard  way  of  measuring  some  attribute  of  the  risk  management  process. 
Risk  and  mitigation  plan  metrics  can  be  qualitative  or  quantitative. 

mitigate-  A  mitigation  approach  that  deals  with  a  risk  by  developing  strategies  and 
actions  for  reducing  (or  eliminating)  the  impact,  probability,  or  both,  of  the  risk  to  some 
acceptable  level.  It  may  also  involve  shifting  the  timeframe  when  action  must  be  taken. 
(See  mitigation  plan.) 

mitigation  approach-The  approach  taken  to  deal  with  a  risk.  This  can  be  to  accept  it, 
research  it,  watch  it,  or  mitigate  it. 

mitigation  costs-Those  costs  directly  associated  with  mitigating  specific  risks  to  the 
project.  This  is  the  cost  of  carrying  out  the  mitigation  plan.  See  infrastructure  costs  and 
risk  management  costs. 
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mitigation  plan-An  action  plan  for  risks  that  are  to  be  mitigated.  It  documents  the  strat¬ 
egies,  actions,  goals,  schedule  dates,  tracking  requirements,  and  all  other  supporting 
information  needed  to  carry  out  the  mitigation  strategy. 

offeror-A  contractor  who  submits  a  proposal  in  response  to  a  solicitation  package. 
optimizing  level-(See  maturity  leveL)g 

organization-The  parent  organization  of  the  acquisition  organization. 

organization’s  measurement  program-The  set  of  related  elements  for  addressing  an 
organization’s  measurement  needs.  It  includes  the  definition  of  organization-wide  mea¬ 
surements,  methods  and  practices  for  collecting  organizational  measurements  and  ana¬ 
lyzing  data,  and  measurement  goals  for  the  organization. 

orientation-An  overview  or  introduction  to  a  topic. 

periodic  review-A  review  that  occurs  at  specified  regular  time  intervals.  (See  event- 
driven  basis  for  contrast.) 

Plan— One  of  the  six  functions  of  the  SEI  risk  management  paradigm.  The  Plan  function 
is  a  process  for  determining  what,  if  anything,  should  be  done  with  a  risk.  It  produces  an 
action  plan  for  individual  or  sets  of  related  risks.  Planning  answers  the  questions 

•  Is  it  my  risk?  (responsibility) 

•  What  can  I  do?  (approach) 

•  How  much  and  what  should  I  do?  (scope  and  actions) 

probability-The  likelihood  the  risk  will  occur.  Probability  is  one  of  the  three  attributes 
of  a  risk. 

policy-A  guiding  principle,  typically  established  by  senior  management,  that  is  adopted 
by  an  organization  or  project  to  influence  decisions. 

prime  contractor-An  individual,  partnership,  corporation,  or  association  that  adminis¬ 
ters  a  subcontract  to  design,  develop,  and/or  manufacture  one  or  more  products. 

procedure-A  written  description  of  a  course  of  action  to  be  taken  to  perform  a  given 
task. 

processh-A  set  of  activities  performed  for  a  given  purpose  (e.g.,  the  software  acquisition 
process). 

process  capability-The  range  of  expected  results  that  can  be  achieved  by  following  a 
process.  (See  process  performance  for  contrast.) 

process  capability  baseline-A  documented  characterization  of  the  range  of  expected 
results  that  would  normally  be  achieved  by  following  a  specific  process  under  typical 
circumstances.  A  process  capability  baseline  is  typically  established  at  an  organiza¬ 
tional  level.  (See  process  performance  baseline  for  contrast.) 


process  descriptions-Documentation  that  specifies,  in  a  complete,  precise,  verifiable 
manner,  the  requirements,  design,  behavior,  or  other  characteristics  of  a  process.  It  may 
also  include  the  procedures  for  determining  whether  these  provisions  have  been  satis¬ 
fied. 

process  measurement-The  set  of  definitions,  methods,  and  activities  used  to  take  mea¬ 
surements  of  a  process  and  its  resulting  products  for  the  purpose  of  characterizing  and 
understanding  the  process. 

process  performance-A  measure  of  the  actual  results  achieved  by  following  a  process. 
(See  process  capability  for  contrast.) 

process  performance  baseline-A  documented  characterization  of  the  actual  results 
achieved  by  following  a  process.  A  process  performance  baseline  is  typically  estab¬ 
lished  at  the  project  level,  although  the  initial  process  performance  baseline  will  usually 
be  derived  from  the  process  capability  baseline.  (See  process  capability  baseline  for 
contrast.) 

project-An  undertaking  that  is  focused  on  acquiring  a  specific  product.  The  product 
may  include  hardware,  software,  and  services.  Typically,  a  project  has  its  own  funding, 
cost  accounting,  and  delivery  schedule. 

project  manager-The  role  with  total  business  responsibility  for  an  entire  project-,  the 
individual  who  directs,  controls,  administers,  and  regulates  a  project  acquiring  software, 
a  hardware/software  system,  or  services.  The  project  manager  is  the  individual  ulti¬ 
mately  responsible  to  the  end  user. 

project  office-The  aggregate  of  individuals  assigned  the  primary  responsibility  for  soft¬ 
ware  acquisition  in  the  contracted  effort.  A  project  office  may  vary  in  size  fi'om  a  single 
individual  assigned  part  time  to  a  large  organization  assigned  full  time. 

project  team-All  individuals  that  have  an  assigned  software  acquisition  responsibility 
in  the  contracted  effort.  A  project  team  may  vary  in  size  from  a  single  individual 
assigned  part  time  to  a  large  organization  assigned  full  time. 

project’s  defined  software  acquisition  process-(See  software  acquisition  process.) 

quantitative  control-Any  quantitative  or  statistically  based  technique  appropriate  to 
analyze  a  software  acquisition  process,  identify  special  causes  of  variations  in  the  per¬ 
formance  of  the  software  acquisition  process,  and  bring  the  performance  of  the  software 
acquisition  process  within  well-defined  limits. 

quantitative  IeveI-(See  maturity  level.) 

repeatable  leveI-(See  maturity  level.) 

required  training-Training  required  by  the  acquisition  organization.  (See  training  for 
contrast.) 

research— A  mitigation  approach  that  involves  investigating  the  risk  itself  to  increase 
the  level  of  understanding  until  a  decision  about  what  to  do  with  the  risk  can  be  reached. 
This  is  a  preliminary  approach  used  to  make  sure  an  informed  decision  can  be  made  to 
accept,  watch,  or  mitigate  a  risk. 
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risk-The  possibility  of  suffering  loss.  In  a  development  project,  the  loss  describes  the 
impact  to  the  project,  which  could  be  in  the  form  of  diminished  quality  of  the  end  prod¬ 
uct,  increased  costs,  delayed  completion,  or  failure. 

risk  management-The  process  associated  with  identifying,  analyzing,  planning,  track¬ 
ing,  and  controlling  project  risks. 

risk  management  costs-The  costs  associated  with  performing  risk  management  activi¬ 
ties — e.g.,  identifying  risks,  building  status  reports,  and  developing  mitigation  plans. 
This  should  not  be  confused  with  mitigation  costs  or  infrastructure  costs. 

role-A  unit  of  defined  responsibilities  that  may  be  assumed  by  one  or  more  individuals. 

software  acquisition  management  personnel-Those  individuals  who  are  trained,  edu¬ 
cated,  or  experienced  in  software  acquisition  management  and  who  are  either  assigned 
to  or  support  the  project  team  in  the  performance  of  software  acquisition  activities. 

software  acquisition  plans-The  collection  of  plans,  both  formal  and  informal,  used  to 
express  how  software  acquisition  activities  will  be  performed;  for  example,  the  Software 
Acquisition  Risk  Management  Plan  or  Project  Management  Plan. 

software  acquisition  process-A  set  of  activities,  methods,  practices,  and  transforma¬ 
tions  that  people  use  to  acquire  software  and  the  associated  products. 

•  acquisition  organization’s  standard  software  acquisition  process-The  acquisition 
organization's  fundamental  software  acquisition  process  which  guides  the 
establishment  of  each  project's  defined  software  acquisition  process. 

•  project’s  defined  software  acquisition  process-The  project’s  tailored  version  of  the 
acquisition  organization’s  standard  software  acquisition  process. 

software  acquisition  process  assets-A  collection  of  entities,  maintained  by  an  organi¬ 
zation,  for  use  by  projects  in  developing,  tailoring,  maintaining,  and  implementing  their 
software  acquisition  process. 

Some  examples  of  these  software  acquisition  process  assets  include 

•  the  acquisition  organization's  standard  software  acquisition  process 

•  descriptions  of  the  software  life  cycles  approved  for  use 

•  the  guidelines  and  criteria  for  tailoring  the  acquisition  organization's  standard  software 
acquisition  process 

•  the  organization’s  software  acquisition  process  database 

•  a  library  of  software  acquisition  process-related  documentation 

Any  entity  that  the  organization  considers  useful  in  performing  the  activities  of  process 
definition  and  maintenance  could  be  included  as  a  process  asset. 

software  acquisition  process  group-This  group  is  responsible  for  the  definition, 
improvement,  and  maintenance  of  the  acquisition  organization's  standard  software 
acquisition  process  and  related  process  assets,  including  guidelines  for  all  projects  to 
tailor  iht  standard  software  acquisition  process  to  their  specific  situations.  It  coordinates 
process  activities  with  the  software  projects  and  related  elements  of  the  organization. 


software  acquisition  process-related  documentation-Documents  and  document  frag¬ 
ments  that  may  be  of  use  to  future  project  teams  when  tailoring  the  acquisition  organi¬ 
zation's  standard  software  acquisition  process.  The  examples  may  cover  subjects  such 
as  a  project's  defined  software  acquisition  process,  standards,  procedures,  software 
acquisition  risk  management  plans,  and  training  materials. 

software  acquisition  process  repository-A  collection  of  software  acquisition  process 
information  (e.g.,  estimated  and  actual  data  on  software  project  size,  effort,  and  cost; 
and  project  team  productivity  and  quality  data)  gathered  from  the  software  acquisition 
projects  that  is  maintained  by  the  acquisition  organization  to  support  its  software  acqui¬ 
sition  definition  and  improvement  activities. 

software  acquisition  project-An  undertaking  that  is  focused  on  acquiring  the  software 
components  and  associated  documentation  of  a  system.  A  software  project  may  be  part 
of  a  project  building  a  hardware/software  system. 

software  acquisition-related  group-A  collection  of  individuals  (both  managers  and 
technical  staff)  representing  a  software  discipline  that  supports,  but  is  not  directly 
responsible  for,  software  acquisition.  Examples  of  software  disciplines  include  software 
configuration  management  and  software  quality  assurance. 

software  acquisition  risk  management  plan-A  formal  plan  or  documentation  of  the 
risk  management  practice  (processes,  methods,  and  tools)  to  be  used  for  a  specific 
project.  This  directs  and  manages  the  activities  used  to  perform  risk  management  within 
that  project. 

software  life  cycle-The  period  of  time  that  begins  when  a  software  product  is  conceived 
and  ends  when  the  software  is  no  longer  available  for  use.  The  software  life  cycle  typi¬ 
cally  includes  a  concept  phase,  requirements  phase,  design  phase,  implementation 
phase,  test  phase,  installation  and  checkout  phase,  operation  and  maintenance  phase, 
and,  sometimes,  retirement  phase. 

solicitation  package-When  seeking  suppliers  for  a  particular  acquisition,  it  is  the  infor¬ 
mation  distributed  which  tells  the  interested  bidders  what  the  requirements  are,  how  to 
prepare  their  proposals,  how  proposals  will  be  evaluated,  and  when  to  submit  their  pro¬ 
posals.  Sometimes  called  request  for  proposals  (RFP). 

Standard-Mandatory  requirements  employed  and  enforced  to  prescribe  a  disciplined, 
uniform  approach  to  software  development  or  acquisition. 

standard  software  acquisition  process-(See  software  acquisition  process.) 

tailor-To  modify  a  process,  standard,  or  procedure  to  better  match  process  or  product 
requirements. 

technology-The  application  of  science  and/or  engineering  in  accomplishing  a  particular 
result. 

timeframe-The  period  when  action  is  required  to  mitigate  the  risk.  Timeframe  is  one  of 
the  three  attributes  of  a  risk. 
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Track-One  of  the  six  functions  of  the  SEI  risk  management  paradigm.  The  Track  func¬ 
tion  is  a  process  in  which  risk  data  are  monitored  by  the  person(s)  responsible  for  track¬ 
ing  watched  and  mitigated  risks. 

Tracking  risks  includes  three  activities: 

•  acquiring  tracking  data 

•  compiling  tracking  data 

•  reporting  tracking  data 

trsaning-Project  team  training.  (See  required  training  for  contrast.) 

watch-A  mitigation  approach  that  monitors  a  risk  and  its  attributes  for  significant 
change.  Watched  risks  may  later  be  mitigated  or  closed  without  any  further  action, 
depending  upon  how  it  changes  as  time  progresses. 
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Software  acquisition  is  the  process  of  buying  software  and  software  services.  Buying 
software  has  become  a  common  event  in  our  society  and  can  be  done  without  even  leav¬ 
ing  the  home  or  workplace.  Software  vendors  use  the  Internet  to  allow  users  to  down¬ 
load  their  products  immediately,  install  them,  and  enjoy  enhanced  computing  capability 
within  minutes  of  purchase.  This  simplistic  view  of  software  acquisition  doesn’t  hold 
when  acquiring  complex  or  custom  systems  dependent  on  software.  Consider  the  F-22 
fighter.  Eighty  percent  of  the  functionality  allowing  it  to  fly  and  perform  the  fighter 
mission  is  dependent  on  software  [STSC  96].  You  can’t  simply  connect  an  F-22  to  the 
Internet  and  download  commercially  available  navigation  control,  weapon  system,  elec¬ 
tronic  warfare,  and  other  complex  software  systems  and  expect  to  maintain  dominance 
over  the  enemy.  Marciniak  and  Reifer  have  helped  narrow  the  definition  of  software 
acquisition  as  follows: 

software  acquisition  is..,the  process  of  managing  the  acquisition  of  custom  soft¬ 
ware  systems.  The  key  discriminators  of  custom  software  are  that  it  cannot  be 
developed  by  a  small  group  of  people  (say  fewer  than  10)  and  that  it  involves  a 
set  of  users  with  direct  interest  in  and  impact  on  system  requirements*'  [Marcin¬ 
iak  90]. 

Seventy-five  percent  of  all  large-scale,  custom  software-intensive  systems  fail  [Gibbs 
94].  The  primary  cause  of  failure  is  poor  management  on  the  part  of  the  developer  and 
the  acquirer  [STSC  96],  not  lack  of  technical  performance.  Even  with  this  track  record, 
the  demand  for  software-intensive  systems  has  been  growing  consistently  and  steadily. 
More  and  more,  software  costs  dominate  these  systems.  The  Department  of  Defense 
(DOD)  estimated  in  1995  that  software  accounted  for  $35.7  billion  of  new  systems 
while  hardware  accounted  for  $6.8  billion  [STSC  96].  On  the  commercial  side,  out¬ 
sourcing  is  estimated  to  reach  $121  billion  by  the  year  2000  [Appleton  96]. 

In  1987,  California’s  Department  of  Motor  Vehicles  (DMV)  decided  it  needed  to  merge 
its  driver  license  and  vehicle  registration  systems.  Seven  years  and  $44.3  million  later, 
the  DMV  cancelled  the  project  after  facing  a  4  year  schedule  slip  and  a  projected 
increase  in  cost  6.5  times  over  what  was  originally  expected  [Gibbs  94]. 

The  software  development  life  cycle  can  be  depicted  as  a  series  of  events  performed 
sequentially,  concurrently,  or  cyclically.  Whichever  approach  is  selected,  Ae  following 
steps  are  usually  performed  when  developing  a  software-intensive  system  and  they  pro¬ 
vide  a  template  for  understanding  software  development  and  its  application  in  more 
sophisticated  development  models  such  as  the  incremental  model  [Pressman  92]. 

•  system  requirements  analysis 

•  software  requirements  analysis 

•  preliminary  design 

•  detailed  design 

•  code  and  unit  test 

•  software  test  and  integration 

•  subsystem  test  and  integration 

•  system  test  and  integration 
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The  software  acquisition  life  cycle  encompasses  the  software  development  life  cycle 
with  additional  activities  performed  on  either  end.  The  Department  of  Defense  defines 
four  acquisition  phases  [DOD  96]: 

•  Phase  0:  Concept  Exploration 

•  Phase  1:  Program  Definition  and  Risk  Reduction 

•  Phase  2:  Engineering  and  Manufacturing  Development 

•  Phase  3:  Production,  Fielding/Deployment,  and  Operational  Support 

For  clarity,  the  life  cycle  is  simplified  into  three  phases:  pre-development  activities, 
development  activities  (the  software  development  process  fits  in  here),  and  post-devel¬ 
opment  activities  as  depicted  in  the  figure  below.  Each  of  these  phases  contain  critical 
activities  performed  by  the  acquisition  organization  which  can  lead  to  a  project’s  suc¬ 
cess  or  failure. 


Pre-Development 

Activities 

Development 

Activities 

Post-Development 

Activities 

The  pre-development  activities  can  include  determining  strategic  business  objectives 
based  on  market,  needs,  or  threat  analysis,  developing  proposed  alternatives  to  meet 
those  objectives,  performing  feasibility  studies,  capturing  system  requirements,  estimat¬ 
ing  project  costs,  capturing  and  continuously  managing  risks,  selecting  one  or  more 
projects  to  fund,  deciding  to  make  or  buy  the  new  capability,  staffing  a  project  office, 
developing  a  solicitation  package,  selecting  a  contractor,  and  teaming  with  the  winner  to 
start  the  project.  While  the  list  is  by  no  means  exhaustive,  each  of  these  activities  is 
critical  to  the  project’s  eventual  success. 

During  development,  the  acquirer  tracks  and  oversees  the  contracted  effort,  maintains 
the  health  of  the  project  team,  approves  design  decisions,  and  is  the  liaison  between  the 
user  and  the  developer.  Many  custom,  software-intensive  systems  take  years  to 
develop.  This  presents  the  acquirer  with  two  major  problems;  knowing  the  real  status 
of  the  project  at  any  given  time  and  struggling  wifh  the  need  to  freeze  requirements  in  a 
changing  environment. 

Post-development  activities  can  include  user  acceptance,  installation  of  the  new  capa¬ 
bility  in  the  operational  environment,  and  transitioning  maintenance  to  the  software 
support  organization.  User  rejection  is  a  very  real  possibility.  Finding  out  that  users 
have  rejected  the  system  during  this  phase  is  a  major  failure  on  the  part  of  the  acquisi¬ 
tion  organization.  This  risk  can  be  mitigated  “...with  intimate  user  involvement,  and 
often  with  periodic  prototype  or  early  version  field  tests”  [STSC  96]. 

Contracting  for  enhancements,  defect  removal,  and  adaptation  can  be  seen  as  mini¬ 
acquisitions  requiring  another  cycle  starting  with  the  pre-development  activities. 
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Making  it 
Successful 


Given  the  activities  performed  by  the  acquisition  organization  and  the  potential  for 
project  disaster  based  on  mismanagement,  the  need  for  software  acquisition  improve¬ 
ment  becomes  obvious.  Appendix  B,  p.  75  introduces  the  Software  Acquisition  Capa¬ 
bility  Maturity  Model  (SA-CMM),  which  can  be  used  to  help  an  organization  baseline 
software  acquisition  capability  and  define  an  improvement  path  wiA  the  goal  of  stan¬ 
dardizing  the  way  the  organization  does  business,  increasing  predictability,  and  reduc¬ 
ing  risk. 
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Appendix  B 

The  Software  Acquisition  CMM 

During  the  acquisition  of  a  custom  software  system,  the  acquirer  and  the  developer  have 
distinct  responsibilities.  The  acquirer  acts  as  an  agent  for  an  end  user  while  the  devel¬ 
oper  responds  to  the  acquirer’s  requirements  and  delivers  a  specified  capability  [Marcin- 
i^  90].  Each  have  an  important  role  to  play  in  acquiring  a  software-intensive  system. 
The  following  table  contains  examples  of  the  types  of  responsibilities  each  may  have 
when  acquiring  a  software  system  and  is  not  meant  to  be  comprehensive. 

Role  Responsibilities 

Acquirer  Describe  functional  requirements 

Manage  the  user’s  needs 
Allocate  resources 
Contract  with  developer 
Staff  project  team 

Monitor  development  (cost,  schedule,  performance) 
Determine  support  requirements 

Developer  Decide  to  bid 

Staff  a  proposal  team 

Estimate  resources 

Develop  detailed  proposal 

Respond  to  acquirer  post-contract  award 

Analyze  and  allocate  requirements 

Design  solution 

Develop  software 

Test  and  install  capability 

Hand  maintenance  over  to  support  agency 


Software  acquisition  process  capability  is  the  ability  of  an  organization’s  acquisition 
process  to  produce  predictable  and  consistent,  planned  results.  The  Capability  Maturity 
Model  for  Software  (SW-CMM)  has  been  used  for  several  years  to  help  software  devel¬ 
opers  increase  their  software  process  capability  [Paulk  95].  By  using  the  SW-CMM  to 
help  define  and  improve  process  capability,  software  development  organizations  have 
seen  as  much  as  an  8:1  return  on  investment  [Herbsleb  94].  The  intangible  benefits  of 
using  a  CMM-based  improvement  program  can  be  even  more  impressive.  Pacific  Bell’s 
benefits  from  using  the  SW-CMM  are  listed  below  [Kolinger  96]. 

Pacific  Bell’s  Benefits  from  Using  the  SW-CMM 

A  repeatable  process  for  commitment  management  is  established. 

Consistent  program  office  practices,  understood  by  all,  reduce  the  PM’s  workload. 
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A  Collection  of 
Key  Practices 


Pacific  BelFs  Benefits  from  Using  the  SW-CMM 

Project  managers  can  focus  on  critical  project  issues. 

Helps  in  understanding  and  forecasting  capability. 

New  project  managers  become  productive  more  quickly. 

A  consistent  set  of  values  is  communicated. 

Developers  experience  far  fewer  interruptions. 

Project  managers  feel  valued,  supported,  and  empowered. 
Gives  clients  and  users  confidence  as  it  improves  credibility. 
Allows  normalization  of  project  metrics. 


The  SW-CMM  is  a  collection  of  key  practices  that  developers  can  use  to  baseline  and 
improve  their  software  development  process  capability.  Acquirers  have  a  similar  need 
to  assess  the  maturity  of  their  software  acquisition  process  with  the  goal  of  identifying 
areas  needing  improvement.  The  Software  Acquisition  Capability  Maturity  Model  (SA- 
CMM)  was  developed  by  a  group  of  software  acquisition  experts  from  government  and 
industry  to  address  this  need  [Fisher  99].  The  SA-CMM  is  a  collection  of  key  practices 
formed  into  an  improvement  framework  depicting  organizational  maturity.  The  model 
has  five  maturity  levels  with  key  process  areas  (KPAs)  grouped  at  each  level.  As  an 
organization  masters  the  KPAs  at  a  given  level  and  progresses  from  an  ad  hoc  acquisi¬ 
tion  organization  to  an  organization  embracing  continuous  improvement,  risk  and 
rework  are  reduced. 
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Reducing  Risk 
Through  Process 
Improvement 


Organizations  at  Level  1  of  the  SA-CMM  are  immature.  As  an  organization  progresses 
through  the  model,  its  software  acquisition  process  capability  matures.  Organizations  at 
lower  levels  of  maturity  are  reactionary.  Managers  and  practitioners  make  up  processes 
on  the  fly  and  usually  in  response  to  a  major  problem.  There  is  no  common  understand¬ 
ing  of  how  to  conduct  business,  and  project  members  have  to  re-invent  the  wheel  at 
every  turn.  Project  managers  have  little  insight  into  the  activities  of  the  project  team  and 
are  usually  caught  by  surprise  when  the  contractor  or  project  team  encounters  problems. 

A  mature  software  acquisition  organization,  by  contrast,  is  proactive.  It  has  the  ability 
to  capture,  maintain,  and  improve  software  acquisition  processes.  These  processes  are 
communicated  to  existing  staff  and  new  employees.  The  organization’s  processes  are 
documented,  usable,  and  reflect  how  work  actually  gets  done.  Roles  and  responsibilities 
are  defined  within  the  process  and  are  clear  across  the  project  and  entire  organization. 
Managers  monitor  and  evaluate  the  quality  of  the  software  acquisition  products  and  the 
process  that  produces  them.  Cost  and  schedule  estimates  are  developed  based  on  histor¬ 
ical  data  and  the  project  achieves  realistic  expectations  [Paulk  95]. 

KPAs  are  grouped  at  maturity  levels  within  the  SA-CMM.  A  KPA  is  a  cluster  of  related 
practices  in  an  area  of  software  acquisition.  When  these  practices  are  performed  collec¬ 
tively,  they  achieve  the  goals  that  establish  process  capability  in  that  area  [Fisher  99]. 

To  master  a  KPA,  its  goals  must  be  achieved.  Supporting  those  goals  are  tJhe  institution¬ 
alization  features  which  help  install  the  KPA  as  a  common  business  practice  and  the 
activities  which  are  performed  by  the  acquisition  organization  to  accomplish  the  goals. 


A  mature  organization  with  well-defined  processes  can  expect  improvement  in  predict¬ 
ability,  control,  and  effectiveness  [Paulk  95].  Improvements  in  these  three  areas  will 
reduce  the  risks  associated  with  acquiring  software.  See  Appendix  E,  p.  101  for  a  dis¬ 
cussion  of  the  appraisal  process  used  to  determine  an  acquisition  organization’s  software 
acquisition  capability  using  the  SA-CMM. 
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Risk  Management  Methods  and  Tools 

This  appendix  provides  a  summary  of  the  methods  and  tools  that  can  be  used  to  perform 
the  risk  management  paradigm  functions.  Methods  are  systematic  approaches  to  per¬ 
forming  risk  management  processes  while  tools  are  templates  or  forms.  More  detailed 
information  for  all  of  the  methods  and  tools  in  this  section  are  featured  in  the  Continu¬ 
ous  Risk  Management  Guidebook  [Dorofee  96].  The  project  team  may  select  these  or 
other  methods  and  tools  to  perform  acquisition  risk  management. 
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Identification 
Methods  and 
Tools 


The  following  table  provides  a  summary  of  some  of  the  methods  and  tools  that  can  be 
used  to  support  the  components  of  risk  identification.  Additional  references  are  pro¬ 
vided  for  some  of  the  methods  and  tools.  Any  identification  method  or  tool  which  is  not 
listed  in  the  table  below,  but  which  a  project  team  feels  is  appropriate  for  its  needs,  can 
be  selected  by  a  project  team  as  needed. 


Component 

Method  or  Tool 

Description 

All  components 

Risk  information  sheet 

A  form  that  documents  information  about  a  risk,  similar  to  a 
software  trouble  or  problem  report.  As  information  is 
acquired  or  collected,  it  is  added  to  the  risk  information 
sheet  (see  Appendix  D,  p.  93). 

Capture  statement  of 
risk 

Brainstorming 

A  method  where  project  personnel  verbally  identify  risks  as 
they  think  of  the  risks.  This  method  provides  the 
opportunity  for  participants  to  build  on  each  others’  ideas. 
[Lumsdaine  90,  Osborn  53,  Xerox  92]. 

Periodic  risk  reporting 

A  method  requiring  mandatory  and  scheduled  reporting  of 
risks  by  project  personnel. 

Project  profile 
questions 

A  tool  used  to  tailor  the  taxonomy-based  questionnaire 
(TBQ)  based  on  project  characteristics. 

Risk  form 

A  form  used  to  document  new  risks  as  they  are  identified, 
(see  Appendix  D,  p.  93) 

Short  TBQ 

A  shortened  version  of  the  TBQ  which  can  be  used  in 
conjunction  with  voluntary  or  periodic  risk  reporting.  It  can 
also  be  used  during  meetings  and  one-on-one  interviews  to 
help  identify  risks. 

Taxonomy-based 
questionnaire  (TBQ) 

A  listing  of  interview  questions  which  are  organized 
according  to  the  software  development  risk  taxonomy 
[Carr  93]. 

TBQ  interviews 

A  method  where  structured  peer  group  interviews  are 
conducted  using  the  TBQ. 

Voluntary  risk 
reporting 

A  method  where  project  personnel  voluntarily  submit  risk 
forms  whenever  new  risl^  are  identified. 

Capture  context  of  risk 

All  above  methods  and 
tools 

All  of  the  above  methods  and  tools  are  applicable  to 
capturing  the  context  of  a  risk  because  context  is  required 
any  time  a  risk  is  identified. 
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Analysis 
Methods  and 
Tools 


The  following  table  provides  a  summary  of  some  of  the  methods  and  tools  that  can  be 
used  to  support  the  components  of  risk  analysis.  Additional  references  are  provided  for 
some  of  the  methods  and  tools.  Any  analysis  method  or  tool  which  is  not  listed  in  the 
table  below,  but  which  a  project  team  feels  is  appropriate  for  its  needs,  can  be  selected 
by  a  project  team  as  needed. 


Component 

Method  or  Tool 

Description 

All  components 

Risk  information  sheet 

A  form  that  documents  information  about  a  risk,  similar  to  a 
software  trouble  or  problem  report.  As  information  is 
acquired  or  collected,  it  is  added  to  the  risk  information 
sheet  (see  Appendix  D,  p.  93). 

Evaluate 

Binary  attribute 
evaluation 

A  method  where  each  risk  is  evaluated  with  respect  to 

•  impact  (significant,  insignificant) 

•  probability  (likely,  unlikely) 

•  timeframe  (near-term,  far-term) 

Note:  Each  attribute  only  has  two  possible  values. 

Risk  form 

A  form  that  can  be  used  to  capture  the  results  of  the  binary 
attribute  evaluation  or  tri-level  attribute  evaluation  methods 
for  a  risk  (see  Appendix  D,  p.  93). 

Tri-level  attribute 
evaluation 

A  method  where  each  risk  is  evaluated  with  respect  to 

•  impact  (catastrophic,  critical,  marginal) 

•  probability  (very  likely,  probable,  improbable) 

•  timeframe  (imminent,  near-term,  far-term) 

Note:  Each  attribute  only  has  three  possible  values 
[Air  Force  88,  Sisti  94]. 

Classify 

Affinity  grouping 

A  method  where  risks  that  are  naturally  related  are  grouped 
together.  The  concept  that  ties  together  the  risks  in  the 
group  is  also  identified  [Brassard  89,  Brassard  94]. 

Bar  graph 

A  tool  that  presents  a  graphical  summary  of  the  number  of 
risks  in  each  classification  category  [Brassard  89,  Hays  88, 
Moran  90]. 

Risk  form 

A  form  used  to  capture  the  results  of  the  affinity  grouping  or 
taxonomy  classification  methods  for  a  risk  (see  Appendix 
D,p.93). 

Taxonomy 

classification 

A  method  that  groups  risks  according  to  software 
development  areas  using  the  software  development  risk 
taxonomy’s  class/element/attribute  structure  [Carr  93]. 
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Component 

Prioritize 


Method  or  Tool 

Description 

Comparison  risk 
ranking 

A  method  where  risks  are  ranked.  The  method  is  conducted 
by  comparing  two  risks  to  each  other,  based  on  an 
established  criterion  or  set  of  criteria.  The  comparisons 
continue  until  every  risk  has  been  compared  to  all  of  the 
other  risks  [Fitzgerald  90a,  Fitzgerald  90b,  Xerox  92]. 

Multivoting 

The  multivoting  method  is  a  general  voting  technique  to 
select  the  most  important  items  on  a  list.  For  a  large  list,  a 
series  of  votes  is  used  to  reduce  the  list  to  a  workable 
number.  Each  participant  is  given  a  number  of  votes  to  be 
distributed  across  the  items  on  the  list.  Participants  vote 
individually,  and  the  votes  are  tallied  by  one  of  the  group 
members  [Scholtes  88,  Xerox  92]. 

Pareto  top  N 

A  method  where  the  most  important  risks  to  the  project  are 
selected  based  on  the  results  of  the  tri-level  attribute 
evaluation  [Juran  89]. 

Potential  top  N 

A  method  where  the  most  important  risks  to  the  project  are 
selected  based  on  individual  opinions,  which  are  surfaced 
using  the  top  5  method.  All  of  the  participants’  number  one 
risks  are  grouped  together;  then,  all  of  the  number  two  risks 
are  grouped  together.  This  continues  until  all  of  the  risks  in 
the  participants’  top  5  lists  are  placed  in  a  group.  If  a  risk 
appears  in  more  than  one  group,  it  is  eliminated  from  all  but 
the  highest  ranking  group.  The  result  is  a  non-ordered  list  of 
important  risks  to  the  project. 

Top  5 

A  method  where  individuals  choose  the  top  5  risks  to  the 
project  as  part  of  a  group  analysis  effort,  such  as  the 
potential  top  N  method.  The  intent  is  to  collect  individual 
perspectives  on  which  risks  are  important  to  the  project. 
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The  following  table  provides  a  summary  of  some  of  the  methods  and  tools  that  can  be 
used  to  support  the  components  of  planning.  Additional  references  are  provided  for 
some  of  the  methods  and  tools.  Any  planning  method  or  tool  which  is  not  listed  in  the 
table  below,  but  which  a  project  team  feels  is  appropriate  for  its  needs,  can  be  selected 
by  a  project  team  as  needed. 


Component 

Method  or  Tool 

Description 

All  components 

Planning  decision 
flowchart 

A  tool  that  can  be  used  to  remind  planners  of  possible 
planning  approaches  as  well  as  the  criteria  for  selecting 
those  approaches  (see  Appendix  D,  p.  93). 

Risk  information  sheet 

A  form  that  is  used  to  document  the  chosen  mitigation 
strategy  and  actions  as  well  as  who  has  responsibility  for 
developing  the  plan  (see  Appendix  D,  p.  93). 

Assign  responsibility 

No  specific  method  or 
tool 

Assigning  responsibility  is  a  management  or  team  decision. 

Determine  approach 

Goal-question-metric 

A  method  where  metrics  are  identified  to  track  the  progress 
of  a  mitigation  strategy  and  the  changes  in  the  status  of  a 
risk.  A  list  of  questions  is  developed  and  used  to  structure  a 
brainstorming  session  to  identify  appropriate  metrics 
[Basili  84,  Pulford  96,  Grady  92]. 

Define  scope  and 
actions 

Action  item  list 

A  document  that  lists  one  or  more  simple  actions  required  to 
mitigate  a  risk.  The  status  of  the  mitigation  effort  is  tracked 
and  reported  when  using  this  tool. 

Planning  worksheet 

A  form  that  is  used  to  identify,  analyze,  and  document 
alternative  mitigation  actions  and  decisions.  It  also  serves 
as  a  historical  record  of  the  alternatives  that  were  considered 

before  the  mitigation  plan  was  chosen  (see  Appendix 

D,p.  93). 
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Component  Method  or  Tool  Description 

Define  scope  and  Problem-solving  A  method  where  task  plans  are  developed  to  mitigate  a  risk 

actions  (cont.)  planning  or  risk  set.  This  method  is  used  for  a  complex  risk  or  set  of 

related  risks  where  dependencies  are  high  and  mitigation 
may  be  costly.  Management  approval  is  likely  required  to 
implement  the  task  plan.  Often,  group  expertise  is  required 
to  develop  the  detailed  plans  and  schedules,  and  this  method 
is  designed  for  groups. 

Problem-solving  planning  includes  the  following  methods 
and  tools: 

•  affinity  grouping 

•  brainstorming 

•  cause  and  effect  analysis 

•  cost-benefit  analysis 

•  Gantt  charts 

•  goal-question-metric 

•  list  reduction 

•  multivoting 

•  PERT  charts 

•  work  breakdown  structure 

[Kepner  81,  Lumsdaine  90,  Scholtes  88,  Xerox  92] 

Risk  form  A  form  that  provides  a  field  to  document  the  recommended 

mitigation  action  (see  Appendix  D,  p.  93). 
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Tracking  There  are  not  many  tools  specifically  designed  for  tracking  risks.  Rather,  there  are 

Approaches  approaches  for  tracking  risks  which  utilize  existing,  general  methods  and  tools.  The  fol¬ 

lowing  table  provides  a  sununary  of  some  of  the  approaches  that  can  be  used  to  support 
the  components  of  risk  tracking.  Any  tracking  approach  which  is  not  listed  in  the  table 
below,  but  which  a  project  team  feels  is  appropriate  for  its  needs,  can  be  selected  by  a 
project  team  as  needed. 


Component  Approach  Description 

Acquire  Re-evaluate  risk  With  this  approach,  the  individual  who  is  responsible  for  the 

attributes  risk  should  periodically  re-evaluate  the  risk  attributes  to 

determine  changes  in  probability,  impact,  and  timeframe. 
Access  to  knowledgeable  individuals  or  other  data  may  be 
required.  This  approach  provides  status  information  for 
watched  risks  and  mitigation  plans.  The  following  methods 
are  used  to  evaluate  risk  attributes: 

•  binary  attribute  evaluation 

•  tri-level  attribute  evaluation 


Direct  communication  This  approach  consists  of  informal  communication  with  the 
personnel  closest  to  the  risk  or  risk  mitigation  activity. 
Often,  the  software  engineers  working  on  the  project  or 
other  personnel  directly  responsible  for  risk  or  plan  actions 
are  interviewed.  In  some  cases,  the  individual  who  is 
interviewed  may  be  the  manager  responsible  for  the  risk  or 
mitigation  plan. 

This  approach  involves  looking  at  the  technical  aspects  of 
the  development  effort’s  progress.  Reviewing  reports  and 
documents  can  be  useful  for  technical  risks  but  can  also 
provide  insight  into  general  project  issues.  This  approach 
can  also  be  used  to  look  for  new  risk  information. 


Review  status  reports  This  approach  involves  reviewing  documentation  that  is 
available  from  the  routine  project  status  meetings.  These 
reviews  can  provide  insight  into  general  project  issues  as 
well  as  status  information  for  watched  risks  and  mitigation 
plans. 

This  approach  involves  using  commercially-available  tools 
to  track  and  collect  progress  and  quality  metrics  from  the 
project’s  products  and  reports,  providing  consistent,  often 
quantitative  risk  data.  The  data  collected  can  be  used  to 
track  risks  and  the  progress  of  mitigation  efforts. 

Compile  Mitigation  plan  This  approach  uses  summaries  or  reports  that  show 

summaries  mitigation  plan  progress.  Mitigation  status  summaries  are 

used  to  support  decisions.  The  following  method  is 
designed  to  convey  information  about  the  status  of  a 
mitigation  plan: 

•  mitigation  status  report 


Automated  data 
collection 


Review  documents 
and  reports 
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Component 

Approach 

Description 

Compile  (cont.) 

Risk  status  summaries 

This  approach  uses  summary  tables,  which  are  concise 
tabular  compilations  of  key  data  items.  The  following 
methods  and  tools  are  designed  to  produce  and  use  tabular 
formats: 

•  risk  information  sheet 

•  spreadsheet  risk  tracking 

•  stoplight  chart 

The  analysis  of  current  status  data  can  identify  both  changes 
in  priority  or  the  need  for  outside  help  as  well  as  new  risks 
to  the  project. 

Trend  summaries 

This  approach  utilizes  graphical  representations  of  compiled 
risk  data.  The  following  are  used  to  present  risk  data  on 
graphs  or  charts: 

•  bar  graph 

•  time  correlation  chart 

•  time  graph 

Report 

Verbal  reporting 

This  approach  uses  informal  means  of  cormnunication  to 
disseminate  risk  data.  The  people  responsible  for  risks  give 
verbal  reports  on  the  general  status  of  their  risks.  This  forum 
can  also  be  used  to  inform  management  of  critical  issues  as 
they  arise  (written  status  would  usually  be  required  as  a 
follow-up).  Verbal  reports  are  useful  for  informal  reporting 
of  status  to  management  as  well  as  immediate  notification  of 
critical  issues  or  changes. 

Written  reporting 

This  approach  can  use  either  formal  or  informal  memoranda 
or  documents  (e.g.,  electronic  mail,  reports).  The  reports 
should  be  integrated  into  the  normal  status  reporting 
mechanisms  used  by  the  organization.  The  following 
methods  can  be  used  to  support  this  activity: 

•  mitigation  status  report 

•  risk  information  sheet 

•  spreadsheet  risk  tracking 

•  stoplight  chart 

Formal  presentations 

This  approach  requires  a  medium  and  format  that  are 
appropriate  for  the  organization.  Formal  presentations  are 
often  supported  by  written  reports  and  contain  additional 
material  tfiat  might  not  be  included  in  written  reports. 
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Tracking 
Methods  and 
Tools 


The  following  table  provides  a  summary  of  some  of  the  methods  and  tools  used  to  sup¬ 
port  the  tracking  approaches  described  above.  Additional  references  are  provided  for 
some  of  the  methods  and  tools.  Any  tracking  method  or  tool  which  is  not  listed  in  the 
table  below,  but  which  a  project  team  feels  is  appropriate  for  its  needs,  can  be  selected 
by  a  project  team  as  needed. 


Component 


Method  or  Tool  Description 


Acquire  Binary  attribute  A  method  where  each  risk  is  evaluated  with  respect  to 

evaluation  •  impact  (significant,  insignificant) 

•  probability  (likely,  unlikely) 

•  timeframe  (near-term,  far-term) 

Note:  Each  attribute  only  has  two  possible  values. 

A  method  where  each  risk  is  evaluated  with  respect  to 

•  impact  (catastrophic,  critical,  marginal) 

•  probability  (very  likely,  probable,  improbable) 

•  timeframe  (imminent,  near-term,  far-term) 

Note:  Each  attribute  only  has  three  possible  values 
[Air  Force  88,  Sisti  94]. 

This  method  requires  compiling  data  using  textual 
information  and  graphics  (e.g.,  time  graphs,  bar  graphs)  to 
document  detailed  information  about  specific  risk 
mitigation  plans.  Mitigation  status  reports  are  used  to 
support  decisions.  The  format  of  the  report  and  the 
information  included  in  the  report  should  be  tailored  to  the 
needs  of  an  organization  [Clark  95]. 

Risk  information  sheet  This  form  documents  information  about  a  risk,  similar  to  a 
software  trouble  or  problem  report.  It  is  used  to  document 
detailed  information  on  specific  risks  and  to  support 
decisions  (see  Appendix  D,  p.  93). 

Spreadsheet  risk  This  method  uses  spreadsheets  to  summarize  the  current 

tracking  statuses  of  all  risks  and  provides  a  way  to  monitor  project 

risks.  The  basic  process  involves  a  periodic  (e.g.,  weekly  or 
monthly)  update  and  review  of  the  risks.  Spreadsheet  risk 
tracking  reports  are  normally  included  as  read-ahead 
material  for  project  meetings,  where  the  reports  are 
reviewed  and  updated  as  appropriate  (see  Appendix 
D,p.  93). 


Compile  and  Report  Mitigation  status 
report 


Tri-level  attribute 
evaluation 
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Component 

Method  or  Tool 

Description 

Compile  and  Report 
(cont.) 

Stoplight  chart 

This  is  a  tool  that  is  used  to  summarize  the  statuses  of 
important  risks  and  their  mitigation  efforts.  The  charts  are 
effective  tools  for  reporting  risk  information  to  senior 
management.  Each  mitigation  plan  is  assigned  one  of  three 
conditions: 

•  green — indicates  that  the  plan  is  working  as  intended  and 
that  no  management  action  is  required 

•  yellow — indicates  that  the  plan  is  not  working  as  intended, 
although  no  management  action  is  required 

•  red— indicates  that  the  plan  is  not  working  and  that 
management  action  is  required  (see  Appendix  D,  p.  93) 

Bar  graph 

This  type  of  graph  depicts  data  across  distinct  categories. 

Bar  graphs  highlight  changes  in  the  number  of  risks  in 
individual  categories  and  can  be  used  to  identify  trends 
[Brassard  89,  Hays  88,  Moran  90]. 

Time  correlation  chart 

This  type  of  graph  shows  the  relationship  of  one  metric  with 
respect  to  another  over  time.  Time  correlation  charts  are 
useful  for  identifying  the  trend  over  time  in  the  relationship 
of  two  metrics  [Brassard  89,  Hays  88,  Moran  90]. 

Time  graph 

This  type  of  graph  illustrates  data  variations  over  time. 

Time  graphs  are  useful  for  identifying  the  trend  over  time  of 
a  risk  metric  [Brassard  89,  Hays  88,  Moran  90]. 
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Control 
Methods  and 
Tools 


Component 

Analyze 


The  following  table  provides  a  summary  of  some  of  the  methods  and  tools  that  can  be 
used  to  support  the  components  of  risk  control  Additional  references  are  provided  for 
some  of  the  methods  and  tools.  Any  control  method  or  tool  which  is  not  listed  in  the 
table  below,  but  which  a  project  team  feels  is  appropriate  for  its  needs,  can  be  selected 
by  a  project  team  as  needed. 

Note:  The  methods  employed  for  risk  control  use  basic  techniques  for  analyzing  and 
deciding  on  an  action,  documenting  the  decision,  and  proceeding  with  the  chosen 
actions.  Most  projects  have  an  established  suite  of  effective  methods  for  these  activities. 
If  such  methods  and  tools  do  exist  within  an  organization,  then  they  should  also  be 
applied  to  risk  data. 


Method  or  Tool 

Description 

Cause  and  effect 
analysis 

This  method  analyzes  the  relationships  and 
interrelationships  between  a  risk  and  it  associated  causes. 
Analyzing  the  causes  and  effects  of  risks  and  actions  may 
provide  additional  insight  into  their  dependencies  and 
relationships  to  support  decisions  [Lumsdaine  90,  Scholtes 
88,  Xerox  92]. 

Cost-benefit  analysis 

This  method  re-evaluates  the  costs  and  benefits  of  a 
particular  mitigation  strategy  if  the  strategy  is  not  having  the 
expected  results.  Cost-benefit  analysis  provides  the 
information  needed  by  decision  makers  to  determine 
whether  to  continue  as  planned  or  to  replan  [Arrow  88, 
Boehm  81,  Xerox  92]. 

Mitigation  status 
report 

This  method  requires  compiling  data  using  textual 
information  and  graphics  (e.g.,  time  graphs,  bar  graphs)  to 
document  detailed  information  about  specific  risk 
mitigation  plans.  Mitigation  status  reports  provide  decision 
makers  wiA  the  data  required  to  determine  the  appropriate 
control  actions  (e.g.,  invoke  contingency  plan,  replan).  The 
format  of  the  report  and  the  information  included  in  the 
report  should  be  tailored  to  the  needs  of  an  organization 
[Clark  95]. 

PERT  Chart 

This  tool  is  a  commonly-used  management  tool  for 
managing  time  and  cost.  PERT  (program  evaluation  and 
review  technique)  charts  are  dependency  and  probability 
schedules  that  can  be  used  to  analyze  the  impacts  of  changes 
in  risk  status  and  mitigation  plans  [Bennatan  92, 

Mayrhauser  90,  Pressman  92,  Xerox  92]. 

Spreadsheet  risk 
tracking 

This  method  uses  spreadsheets  to  summarize  the  current 
statuses  of  all  risks  and  provides  a  way  to  monitor  project 
risks.  The  basic  process  involves  a  periodic  (e.g.,  weekly  or 
monthly)  update  and  review  of  the  risks.  Spreadsheet  risk 
tracking  reports  are  normally  included  as  read-ahead 
material  for  project  meetings,  where  the  reports  are 
reviewed  and  updated  as  appropriate  (see  Appendix 

D,p.  93). 
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Component 

Method  or  Tool 

Description 

Analyze  (cont.) 

Stoplight  chart 

This  is  a  tool  that  is  used  to  summarize  the  statuses  of 
important  risks  and  their  mitigation  efforts.  The  charts  are 
effective  tools  for  reporting  risk  information  to  senior 
management.  Each  mitigation  plan  is  assigned  one  of  three 
conditions: 

•  green — indicates  that  the  plan  is  working  as  intended  and 
that  no  management  action  is  required 

•  yellow — indicates  that  the  plan  is  not  working  as  intended, 
although  no  management  action  is  required 

•  red — indicates  that  the  plan  is  not  working  and  that 
management  action  is  required  (see  Appendix  D,  p.  93) 

Decide 

Closing  a  risk 

This  method  formally  documents  information  about  a  risk 
that  has  been  successfully  mitigated,  has  been  accepted,  or 
has  become  a  problem.  Any  lessons  learned  from  watching 
or  mitigating  the  risk  or  set  as  well  as  the  rationale  for 
closing  the  risk  or  set  should  be  captured  upon  closure. 

List  reduction 

This  method  is  used  with  a  large  number  of  risks,  strategies, 
or  other  ideas.  It  is  especially  useful  when  dealing  with  the 
results  of  a  brainstorming  session  (see  Identification 

Methods  and  Tools,  p.  80).  The  participants  vote  on  each 
item  to  determine  whether  or  not  to  keep  it  on  the  list.  A 
majority  of  votes  generally  keeps  the  item  on  the  list  [Xerox 
92]. 

Multivoting 

The  multivoting  method  is  a  general  voting  technique  to 
select  the  most  important  items  on  a  list.  For  a  large  list,  a 
series  of  votes  is  used  to  reduce  the  list  to  a  workable 
number.  Each  participant  is  given  a  number  of  votes  to  be 
distributed  across  the  items  on  the  list.  Participants  vote 
individually,  and  the  votes  are  tallied  by  one  of  the  group 
members  [Scholtes  88,  Xerox  92]. 

Execute 

Closing  a  risk 

This  method  formally  documents  information  about  a  risk 
that  has  been  successfully  mitigated,  has  been  accepted,  or 
has  become  a  problem. 

See  closing  a  risk  in  the  Decide  component  methods  and 
tools  above. 

Mitigation  status 
report 

This  method  requires  compiling  data  using  textual 
information  and  graphics  to  document  detailed  information 
about  specific  risk  mitigation  plans. 

See  mitigation  status  report  in  the  Analyze  component 
methods  and  tools  above. 

Risk  information  sheet 

A  form  that  documents  information  about  a  risk,  similar  to  a 

software  trouble  or  problem  report.  As  information  is 
acquired  or  collected,  it  is  added  to  the  risk  information 
sheet  (see  Appendix  D,  p.  93). 
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Method  or  Tool  Description 

Spreadsheet  risk  This  method  uses  spreadsheets  to  summarize  the  current 

tracking  statuses  of  all  risks  and  provides  a  way  to  monitor  project 

risks.  Documentation  of  the  action  being  executed  as  well 
as  other  relevant  information  (e.g.,  the  scheduled 
completion  date)  is  added  to  the  spreadsheet. 

See  spreadsheet  risk  tracking  in  the  Analyze  component 
methods  and  tools  above. 

Stoplight  chart  This  is  a  tool  that  is  used  to  summarize  the  statuses  of 

important  risks  and  their  mitigation  efforts.  The  action 
being  executed,  its  current  state  of  success,  and  other 
relevant  information  (e.g.,  the  scheduled  completion  date)  is 
added  to  the  chart. 


See  stoplight  chart  in  the  Analyze  component  methods  and 
tools  above. 
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Overview 


Appendix  D 

Selected  Risk  Management  Forms 

This  appendix  contains  examples  of  selected  risk  management  tools.  All  of  the  tools 
highlighted  in  this  section  are  featured  in  the  Continuous  Risk  Management  Guidebook 
[Dorofee  96].  Examples  of  the  following  forms,  flowcharts,  and  spreadsheets  can  be 
found  in  this  section: 

•  planning  decision  flowchart 

•  planning  worksheet 

•  risk  form 

•  risk  information  sheet 

•  spreadsheet  risk  tracking 

•  stoplight  chart 


Appendix  U 


Keep  I  I  Delegate  I  I  Transfer 


Accept  I  I  Mitigate  I  I  Watch 


Recommendation  for  dealing  with  the  risk  (optlonai); 


Ciassification: 


ID 

Risk  Information  Sheet  Identified: 

Priority 

Statement 

Probability 

Impact 

Origin  Class  Assigned 

to: 

Timeframe 

Context 

Mitigation  strategy 


Contingency  plan  and  trigger 


Status 


Status  date 


Closing  date 


Approval 


Closing  rationale 


Stoplight  Chart 

Status 

Risk 

ID 

Risk 

Statement 

Assigned 

To 

Action  Plan 

Key 

Milestones 

Comments 

RED 

23 

Test  case  development  is 
past  due  and  the 
variability  in  the  level  of 
detail  of  low  level 
requirements  may  result 
in  testability  problems 
and  rework. 

S.  Smith 

Re-evaluate  the  test 
schedules  in  light  of 
current  resources. 

Test  case 
development 
completed  by 
9/15 

Test  case 

development 

will  not  be 

completed 

when 

expected. 

Previously: 

Yellow 

YELLOW  34 


Training  in  tools  and 
processes  has  not  kept 
up  with  needs.  It’s 
taking  longer  to  proceed 
due  to  the  learning 
curve. 


G.  Samms  Institute  weekly  process 
training  sessions  with 
the  software  team. 

Institute  daily  software 
project  reviews  to 
identify  immediate 
issues  and  assign 
mentors. 


50%  of  staff 
through 
training  by  8/ 

14 

75%  of  staff 
through 
training  by  9/1 

100%  of  staff 
through 
training  by  9/ 

15 


Weekly 
training 
sessions  and 
mentor 
assignments 
are  helping  but 
demand  is  still 
more  than  we 
can 

accommodate. 

Previously: 

Red 
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Stoplight  Chart 

Status 

Risk 

ID 

Risk 

Statement 

Assigned 

To 

Action  Plan 

Key 

Milestones 

Comments 

GREEN 

41 

No  system  simulation 
was  done;  we  may  not 
meet  the  performance 
requirements. 

G.  Samms 

Conduct  simulation. 

Simulation 
completed  by 

8/1 

Early 

performance 
tests  meet 
average  2 
second 

response  time. 

Previously: 

Green 
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The  SA-CMM  Appraisal  Process 

Overview  This  appendix  provides  a  summary  of  the  appraisal  process  used  by  the  SEI  when  per¬ 

forming  appraisals  of  an  acquisition  organization’s  acquisition  capability  using  the  SA- 
CMM.  Assessments  and  evaluations  are  discussed  and  a  list  of  typical  appraisal  ques¬ 
tions  for  the  ARM  KPA  are  provided. 
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Appraisal 

Methods 


Appraiser 

Qualifications 


Appraisal 

Diagram 


The  SA-CMM  uses  the  SEI-defined  CMM  appraisal  methods.  The  term  “appraisal”  in 
this  context  is  used  to  address  both  assessments  and  evaluations.  For  assessments,  the 
CMM-Based  Appraisal  for  Internal  Process  Improvement  (CBA-IPI)  method  is  used, 
and  for  evaluations  the  Software  Capability  Evaluation  (SCE)  V3.0  is  used.  Both  meth¬ 
ods  are  CMM  Appraisal  Framework  (CAF)  compliant  [Masters  95]  and  have  been 
approved  and  used  extensively  by  the  SW-CMM  community  the  past  two  years. 

As  with  the  SW-CMM,  SA-CMM  appraisers  must  be  trained  and  qualified  prior  to  exe¬ 
cuting  appraisals.  The  qualifications  for  SA-CMM  appraisers  are  similar  to  that  for  the 
Lead  Assessors  and  Lead  Evaluators  for  the  SW-CMM,  with  acquisition  experience 
being  the  primary  discriminator. 

The  following  diagram  shows  the  generic  appraisal  activities  that  occur  with  SEI CAF- 
compliant  methods. 
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ARM  and  the 

Appraisal 

Process 

Maturity 

Questionnaire 


Typical  ARM 
Questions 


With  respect  to  the  ARM  KPA,  the  activities  and  institutionalization  features  will  be 
appraised  through  interviews  and  documentation  reviews.  Results  are  ultimately 
reported  in  the  form  of  findings  in  the  context  of  the  specific  KPA  they  address.  These 
results  form  the  basis  for  acquisition  process  improvement  planning  and  execution. 

Normally  an  SA-CMM  maturity  questionnaire  will  be  administered  in  advance  of  the 
onsite  appraisal.  This  data  is  used  to  provide  the  appraisal  team  its  first  set  of  organiza¬ 
tion-specific  data  and  may  provide  insight  into  appropriate  tailoring  of  the  onsite  docu¬ 
ment  requests  and  interview  questions  (i.e.,  the  data  gathering  plan). 

The  following  are  typical  questions  that  an  appraiser  may  ask  when  assessing  or  evaluat¬ 
ing  an  organization’s  acquisition  capability: 


Key  Practice 

Question 

Commitment  1 

What  organizational  policy  prescribes  acquisition  risk  management? 

Commitment  2 

Ability  1 

Who  on  this  project  is  responsible  for  coordinating  acquisition  risk 
management  activities? 

Ability  3 

How  were  the  individuals  who  perform  acquisition  risk  management 
activities  chosen? 

Activity  1 

How  does  the  project  ensure  risk  identification,  analysis,  and 
mitigation  activities  are  integrated  into  the  software  acquisition 
planning? 

Activity  2 

Activity  3 

Does  a  Software  Acquisition  Risk  Management  Plan  exist? 

Activity  4 

What  actions  are  taken  by  the  project  team  to  help  encourage  risk 
management  activities? 

Activity  5 

How  does  the  project  ensure  that  risk  identification,  analysis,  and 
mitigation  are  conducted  as  an  integral  part  of  the  solicitation,  project 
performance  management,  and  contract  performance  management 
processes? 

Activity  6 

How  does  the  project  team  track  and  control  risks? 

Activity  7 

Describe  project  reviews  and  how  risk  status  is  integrated. 

Ability  2 

Measurement  1 

Verification! 

Verification  2 

Describe  how  resources  expended  for  acquisition  risk  management 
activities  are  recorded  and  tracked? 

Verification  1 

Verification  2 

How  often  does  the  project  manager  and  acquisition  organization 
management  review  the  acquisition  risk  management  activities? 
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Results  of 
Appraisals 


The  appraisal  final  briefing  and  delivery  of  the  final  report  become  the  basis  for  acquisi¬ 
tion  process  improvement  activities.  Regardless  of  the  determination  of  a  maturity  level 
rating,  the  appraisal  findings  provide  a  rich  source  of  information  upon  which  to  initiate 
and  establish  overall  acquisition  process  improvement  as  well  as  acquisition  risk  man¬ 
agement. 
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